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FACULTY  EFFORT  AND  EFFECTIVENESS 


effectiveness  to  perceived  effort.  Additional  purposes  were 
to  investigate  whether  this  relationship  was  influenced  by 
the  ways  in  which  community  college  faculty  conceived  of 
their  tasks  in  terms  of  predictability,  efficacy,  and 
clarity,  whether  perceived  frequency  of  evaluation  was  more 
strongly  related  to  perceived  effectiveness  or  to  perceived 
effort,  and  whether  perceived  frequency  of  evaluation  in  any 
one  step  of  the  evaluation  model  was  more  strongly  related 
to  leverage.  These  questions  were  investigated  for  these 
six  teaching  tasks:  classroom  teaching,  curriculum  develop- 
ment, professional  development,  student  advising,  college 
service,  and  maintenance  factors. 

A sample  of  500  faculty  members  was  drawn  from  the  28 
community  colleges  in  Florida  using  a probability  propor- 
tionate to  size  sampling  strategy.  A mail-out  questionnaire 
was  used  to  collect  data. 

The  relationship  between  perceived  frequency  of  evalua- 
tion and  perceived  leverage  was  found  to  be  significant  and 
moderately  positive  for  the  tasks  of  curriculum  development 
and  college  service  and  significant  and  moderately  negative 
for  the  task  of  classroom  teaching.  The  strongest  relation- 
ships were  found  for  tasks  that  faculty  perceived  as  low  in 
predictability,  clarity,  and  efficacy.  The  steps  of  the 
evaluation  model  with  the  strongest  relationships  between 


perceived  frequency  of  evaluation  and  leverage  were  allo- 
cating, sampling,  and  planning  for  improvement. 

Based  on  these  data,  community  college  administrators 
might  consider  increases  in  certain  kinds  of  evaluation  for 
as  a way  to  increase  the  leverage  of  faculty. 


CHAPTER  I 


INTRODUCTION 

After  several  decades  of  rapid  expansion  and  growing 
resources,  community  colleges  are  now  facing  two  decades  of 
declining  enrollments,  declining  financial  resources,  and, 
at  the  same  time,  an  increasing  public  emphasis  upon 
accountability  and  the  improvement  of  quality.  This  ground- 
swell  across  the  nation  for  accountability  has  come  from 
taxpayers,  institutional  trustees,  financial  donors,  and 
students,  all  of  whom  are  pressuring  colleges  and  universi- 
ties to  examine  the  cost-effectiveness  of  each  department 
and  the  performance  of  each  professor  tSeldin,  1984). 

At  the  same  time,  academic  departments  are  increasingly 
"tenured-in,”  resulting  in  a less-mobile  and  aging  college 
faculty  and  the  "entrenchment"  of  whole  faculties,  which 
James  Bess  (1982)  called  "economic  and  organizational  incar- 
ceration" (p.  2).  R.  E.  Walton  (1980),  in  a study  of  high 
commitment  work  systems,  anticipated  that  the  results  of 
this  entrenchment  will  be  a shift  from  the  high  motivation 
of  faculty  stemming  from  a strong  sense  of  self-direction 
and  accomplishment  to  neutrality  or  even  apathy.  Some 
faculty  may  become  hostile,  both  privately  and  publicly, 
towards  educational  institutions  which  offer  few  challenges 
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Priest  (1967)  pointed  out  that  "evaluation  is  an 
inherent  element  in  any  organized  attempt  to  achieve  a goal" 
(p.  287).  Kerr  (1976)  described  evaluation  as  the  key  com- 
ponent of  all  MBO  (management  by  objectives)  systems  of 
organizational  planning  and  controlling.  Others,  however, 
contend  that  teaching  is  an  art  and  far  too  complex  to  be 
reduced  to  a set  of  questions  or  too  subtle  to  be  observed 
perceptively  by  outsiders  (Highet,  19S0).  Wilson  (1980), 
commenting  on  the  subjective  nature  of  faculty  evaluation, 
argued  that  "there  is  no  valid  or  reliable  instrument  for 
measuring  teacher  effectiveness,  nor  will  there  ever  be"  (p. 
26). 

Faced  with  the  opposing  forces  of  a pressing  need  to 
develop  evaluation  procedures  and  a general  lack  of  con- 
sensus on  methods,  procedures,  and  desirability  of  evaluat- 
ing faculty,  a plethora  of  research  dealing  at  length  with 
methods  and  procedures  for  assessing  faculty  performance  has 
been  reported  in  the  last  two  decades  (Centra,  1981; 
Millman,  1981).  Researchers  have  investigated,  reported, 
and  summarized  hundreds  of  studies  on  the  validity,  reli- 
ability, sources  of  evidence,  and  on  other  topics  related  to 
the  mechanics  of  faculty  evaluation. 

Evaluation  is  most  often  treated  as  a method  of  achiev- 
ing greater  control  over  the  educational  process,  thereby 
serving  as  a method  for  improving  the  efficiency  and  effec- 
tiveness of  schools  and  colleges.  Little  attention  has  been 


given  to  the  impact  of  such  evaluations  on  faculty  and  thus 
on  students  (Natriello,  1984). 

When  considering  the  effects  of  evaluation,  one  of  the 
first  philosophical  issues  which  arises  is  the  tension 
between  the  concepts  of  academic  freedom  and  academic 
responsibility.  The  right  to  teach,  study,  and  conduct 
research  without  interference,  a concept  implicit  in  the 
principle  of  academic  freedom,  is  often  cited  as  a strong 
argument  against  evaluation  (Doyle,  1982). 

Another  issue  relating  to  the  effects  of  evaluation 
which  has  gained  widespread  attention  in  recent  years  is  the 
effects  of  evaluation  on  the  intrinsic  motivation  of 
faculty.  McKeachie  (1982)  found  in  his  review  of  studies  of 
faculty  motivation  that  faculty  find  their  positions  satis- 
fying in  terms  of  complexity,  autonomy,  interpersonal  rela- 
tionships, and  responsibility  for  other  persons.  Deci 
(1975)  and  White  (1959)  found  that  intrinsic  motivation  is 
based  on  the  innate  need  to  be  competent  and  self- 
determining. 

Opponents  of  evaluation  believe  that  the  use  of 
external  evaluations  undermines  the  intrinsic  motivation  to 
teach  by  decreasing  autonomy  and  self-determination  and  by 
decreasing  opportunities  for  creativity  and  innovation  (Deci 
5 Ryan,  1980).  However,  Morgan  (1984)  indicated  that 
evaluation,  properly  administered,  may  actually  increase 
intrinsic  motivation. 


ity  in 


Sampling  is  the  process  of  collecting  information  on 
the  performance  of  the  allocated  task  according  to  the  cri- 
teria for  performance  that  have  been  set. 

Appraisal  is  the  process  of  bringing  together  the  cri- 
teria set  for  the  task  with  the  sample  of  information  col- 
lected on  that  performance  to  arrive  at  an  evaluation. 

Although  not  explicitly  listed  as  steps  in  the 
Dornbusch  and  Scott  (197S)  model,  two  additional  steps  were 
implicit  in  the  process. 

Communication  is  the  process  of  communicating  the 
results  of  evaluation  to  the  faculty  member. 

Planning  for  improvement  is  the  process  of  developing  a 
plan  to  help  faculty  members  improve  behavior  and  the 
securing  of  resources  for  this  plan. 

In  another  part  of  their  research,  Dornbusch  and  Scott 
(1975)  focused  on  the  variation  of  tasks  performed  by 
faculty  and  suggested  that  different  work  arrangements  and 
supervisory  behaviors  may  be  called  for  depending  upon  the 
task.  They  developed  the  idea  of  "task  conceptions"  to 
refer  to  faculty  beliefs  concerning  the  characteristics  of 
their  tasks  (Dornbusch  5 Scott,  1975). 

Dornbusch  and  Scott  (1975)  suggested  in  their  theory  of 
evaluation  and  authority  that  task  conceptions  will  vary 
along  an  active-inert  dimension.  Tasks  are  variable  in  the 
extent  to  which  goals  can  be  clearly  specified  (clarity),  in 
resistance  to  performing  the  task  is 


which  the 


predictable  (predictability),  and 


means  can  be  developed  for  achieving  the  desired  outcomes 
(efficacy).  Tasks  which  are  high  on  clarity,  predict- 
ability, and  efficacy  are  described  as  inert  tasks,  and 
those  that  are  low  on  these  three  dimensions  are  described 
as  active  tasks.  Dornbusch  and  Scott  (197S)  demonstrated 
that  there  is  a relationship  between  performers'  task  con- 
ceptions and  their  preferences  for  supervisory  and  work 
arrangements.  They  found  that  when  task  conceptions  are 
active,  performers  prefer  to  receive  assignments  in  a way 
that  allows  them  to  make  significant  decisions  about  the 
procedures  for  carrying  out  these  tasks,  but  that  when  task 
conceptions  are  inert,  performers  prefer  to  receive  direc- 
tions that  provide  them  with  predetermined  courses  of 
activity  (Dornbusch  S Scott,  197S). 

Natriello  (1984)  used  a six-step  model  of  the  evalua- 
tion process,  a task-specific  focus,  and  the  concept  of  task 
conceptions,  all  part  of  the  Dornbusch  and  Scott  (197S) 
theory  of  evaluation  and  the  exercise  of  authority,  to 
examine  the  effects  of  evaluation  systems  on  teacher  effort 
and  effectiveness.  He  defined  his  six-step  model  of  the 
evaluation  process  as  the  four-step  Dornbusch  and  Scott 
(1975)  model  plus  the  two  additional  steps  of  communicating 
and  planning  for  improvement,  which  were  implicit  in  the 
Dornbusch  and  Scott  model.  He  chose  not  to  follow  the  lead 
of  earlier  evaluation  researchers  who  used  either  effort  or 
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effort  and  the  level  of  performance.  Miskel,  Defrain,  and 
Wilcox  (1980)  summarized  numerous  studies  and  concluded  that 
the  force  of  motivation  is  significantly  related  to  job 
satisfaction  and  performance.  Vroom's  (1964)  concept  of 
expectancy,  which  is  one  part  of  the  force  of  motivation,  is 
almost  identical  to  Natriello's  (1984)  concept  of  leverage. 

The  use  of  the  concept  of  leverage  shifts  the  focus  of 
managerial  strategies,  including  evaluation,  from  the  goal 
of  increasing  managerial  control  over  efforts  or  outputs  to 
enabling  subordinates  to  gain  control  over  their  own  work 
(Natriello,  1984).  Hackman  and  Oldman  (1980)  described 
control  (experienced  responsibility  for  work  outcomes)  as 
one  of  the  three  psychological  states  necessary  for  main- 
taining intrinsic  motivation.  Argyris  (1962)  listed  control 
as  a necessary  ingredient  for  the  continued  motivation  of 

In  his  study  of  189  middle  school  teachers,  Natriello 
(1984)  hypothesized  that  the  more  frequent  the  evaluation  of 
teaching,  the  greater  will  be  the  leverage  of  teachers  up  to 
the  point  of  overly  close  supervision.  His  second  hypothe- 
sis was  that  the  more  active  (lower  on  clarity,  predictabil- 
ity, and  efficacy)  the  task  conception,  the  greater  the 
impact  of  evaluation  on  leverage  up  to  the  point  of  a task 
being  too  active  for  supervisors  to  make  useful  contribu- 
tions. The  second  hypothesis  was  based  on  the  rationale 
that  because  inert  tasks  are  by  definition  very  predictable 


unlikely 
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and  very  straightforward,  performers  are 
that  increased  supervision  will  increase  the  ratio  of  their 
output  to  their  effort.  In  addition,  because  active  tasks 
are  very  unpredictable  and  very  unclear,  supervisors  may  be 
able  to  make  no  contribution  to  improving  the  ratio  between 
output  and  effort. 

Natriello  (1984)  did  not  confirm  his  first  hypothesis 
because  he  found  a positive  relationship  between  perceived 
frequency  of  evaluation  and  increased  leverage  at  all  levels 
of  evaluation  activities.  There  was  no  decrease  in  the 
relationship  at  high  levels  of  supervision  as  he  had  hypoth- 
esized. He  was  not  able  to  test  his  second  hypothesis, 
because  the  teachers  in  the  study  perceived  all  their  tasks 
to  be  highly  inert  (Natriello,  1984). 

Are  there  implications  in  this  for  community  college 
administrators  striving  to  design  evaluation  systems  and  to 
implement  management  strategies  to  address  the  pressures  and 
problems  discussed  earlier?  While  no  generalizations  can  be 
made  from  data  collected  from  middle  school  teachers, 
Natriello  (1984)  suggested  the  possibility  that  frequency  of 
evaluation  is  related  to  faculty  leverage  (effectiveness/ef- 
fort) at  all  educational  levels  and  that  the  extent  of  this 
relationship  is  dependent  upon  task  conceptions  (active  vs. 
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reasonable  to  conclude  that  if  colleges  are  to  address  the 
problems  of  declining  enrollments,  declining  financial 
resources,  and  an  increasing  public  pressure  for  account- 
ability and  increased  quality,  the  solutions  must  involve 
improving  faculty  effectiveness. 

As  discussed  earlier,  Dornbusch  and  Scott  (1975)  and 
Natriello  (1984)  found  a relationship  between  perceived  fre- 
quency of  evaluation  and  perceived  feelings  of  effectiveness 
in  relationship  to  effort  in  studies  conducted  in  elementary 
schools.  If  an  analysis  of  the  data  collected  from  commun- 
ity college  faculty  substantiates  this  hypothesis,  then  com- 
munity college  administrators  interested  in  enhancing 
faculty  perceptions  of  effectiveness,  and  very  possibly 
actual  effectiveness,  might  consider  increased  formal  and 
informal  evaluation  as  a tool  for  accomplishing  this  goal. 
If  the  hypothesis  is  not  substantiated,  increased  evaluation 
might  be  considered  inappropriate  as  a method  of  increasing 
faculty  perceptions  of  effectiveness  in  relationship  to 
effort.  An  analysis  of  the  data  collected  on  task  concep- 
tions may  also  guide  administrators  as  to  which  of  the  six 
teaching  tasks,  if  any,  are  most  likely  to  be  positively 
affected  by  increased  evaluation. 

If  no  significant  relationship  is  found  between  per- 
ceived frequency  of  evaluation  and  leverage,  those  who 
believe  that  evaluation  is  a threat  to  the  intrinsic  motiva- 
tion of  faculty  and  those  who  view  it  as  a threat  to  the 


principles  of  academic  freedom  will  have  additional  evidence 
on  which  to  base  their  argument. 

However,  if  a significant  relationship  is  found  between 
perceived  frequency  of  evaluation  and  leverage,  this 
research  may  be  used  to  silence  the  opponents  of  evaluation 
who  perceive  it  as  a control  mechanism  limiting  the  autonomy 
and  academic  freedom  of  faculty  (Natriello,  1984).  The 
opponents,  who  have  long  slowed  progress  in  the  area  of 
evaluation,  may  come  to  see  evaluation  as  a positive  prac- 
tice in  enhancing  faculty  control  over  their  teaching  tasks. 
This  research  may  be  used  to  put  in  jeopardy  the  longstand- 
ing philosophy  that  college  faculty  desire  complete  autonomy 
and  that  anything  other  than  complete  autonomy  is  injurious 
to  the  tradition  of  academic  freedom.  It  may  be  a step 
towards  silencing  the  spreading  cry  of  evaluation  opponents 
who  believe  that  evaluation  decreases  intrinsic  motivation 
which  ultimately  decreases  faculty  satisfaction  and  faculty 
effectiveness.  It  may  be  used  to  provide  a new  management 
philosophy  to  community  college  department  chairpersons  who, 
by  tradition  and  training,  have  not  dared  to  "interfere"  in 
the  work  of  faculty. 

Assumptions 

1.  It  was  assumed  that  the  responses  from  the  par- 
ticipants were  honest  and  could  be  taken  at  face  value. 


i predictability,  clarity. 
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Full-time  faculty  member  refers  to  a faculty  member  who 
is  under  contract  for  a full-time  teaching  load  for  at  least 
one  academic  year. 

Inert  task  refers  to  a task  that  a faculty  member  per- 
ceives as  high  on  predictability,  clarity,  and  efficacy. 

Intrinsic  motivation  refers  to  motivation  which  results 
from  an  experience  that  is  an  end  in  itself;  an  activity 
that  is  rewarding  in  itself,  not  just  as  a means  to  an 
external  goal. 

Leverage  refers  to  the  relationship  between  effort  put 
forth  by  a faculty  member  and  the  outcomes  resulting  from 
that  effort  (Natriello,  1984). 

Maintenance  factors  refer  to  the  routine  responsibili- 
ties of  a faculty  member,  such  as  recordkeeping,  office 
hours,  and  required  reporting. 

Perceived  frequency  of  evaluation  refers  to  the  percep- 
tions of  a community  college  faculty  member  as  to  the  fre- 
quency of  formal  and  informal  evaluation  activities  by  their 
immediate  supervisors. 

Task  conception  refer  to  the  ways  faculty  members  view 
the  attributes  of  their  different  tasks  in  terms  of  predict- 
ability, clarity,  and  efficacy  (Dornbusch  6 Scott,  1975). 

Teaching  tasks  refers  to  different  tasks  which  charac- 
terize the  work  of  a community  college  faculty  member.  For 
purposes  of  this  study,  these  tasks  are  defined  as  classroom 
teaching,  curriculum  development,  student  advising. 


professional  development,  college  service,  and 


Procedures 

Sampling  Design 

The  sample  used  for  this  study  was  selected  from  a 
target  population  of  all  full-time  faculty  members  in  the  28 
community  colleges  in  Florida.  A multistage  cluster 
sampling  technique  with  probability  of  cluster  selection 
proportionate  to  size  was  used  so  that  each  faculty  member 
had  an  equal  chance  of  selection.  Using  this  process,  5 of 
the  28  community  colleges  were  selected  and  100  full-time 
faculty  members  were  randomly  selected  from  each  of  the  5 
colleges.  Colleges  containing  less  than  the  required  100 
full-time  faculty  members  were  combined  so  that  each  combin- 
ation contained  the  required  number.  The  total  sample  size 
was  500. 

Collection  of  Data 

The  following  steps  were  followed  to  obtain  the  data 
for  the  study. 

1.  A list  of  the  28  community  colleges  in  Florida  and 
the  number  of  full-time  faculty  members  at  each  was 
developed  using  college  catalogs.  Colleges  with  less  than 
100  faculty  members  were  combined  to  form  one  cluster.  The 
list  was  weighted  using  probability  proportionate  to  size 
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techniques,  and  five  clusters  were  selected  using  computer- 
generated  random  numbers. 

2.  Using  a table  of  random  numbers,  100  full-time 
faculty  members  were  selected  from  each  of  the  five 

3.  A survey  questionnaire  was  constructed  containing 
questions  requiring  Likert-scale  response  categories  which 
resulted  in  summary  measures  of  perceived  frequency  of  eval- 
uation, perceived  leverage,  and  conceptions  of  the  six 
teaching  tasks  along  the  active-inert  continuum. 

4.  The  questionnaire  was  pretested  with  25  faculty 
members  who  were  not  selected  as  part  of  the  sample  to  test 
for  question  clarity.  Modifications  were  made  as  necessary. 

5.  The  questionnaires,  with  stamped  return  envelopes 
and  cover  letters,  were  mailed  to  the  500  faculty  members  in 
the  sample. 

6.  A list  of  non-responders,  plus  duplicate  question- 
naires and  return  envelopes,  was  provided  to  a key  adminis- 
trator at  each  institution  who  had  previously  agreed  to 
serve  as  a coordinator  for  the  study.  Follow-up  continued 
until  82$  of  the  total  sample  had  responded. 

Oata  Analysis 

Using  the  methods  of  Natriello  (1984),  data  obtained 
from  the  survey  responses  were  used  to  calculate  summary 
measures  of  perceived  frequency  of  evaluation,  perceived 
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leverage,  and  conceptions  of  the  six  teaching  tasks  along 
the  active-inert  continuum  according  to  the  degree  of  task 
clarity,  predictability,  and  efficacy. 

For  each  of  the  six  teaching  tasks,  faculty  were 
divided  into  three  groups:  those  reporting  no  stage  of  the 
six-stage  evaluation  model  with  frequent  activity,  those 
reporting  one  stage  with  frequent  activity,  and  those 
reporting  two  or  more  stages  with  frequent  activity. 

Two  measures  of  leverage  were  obtained  from  the  faculty 
surveys.  The  Faculty  Assessment  of  Leverage  (FAL)  was 
obtained  from  an  item  asking  faculty  to  describe  the  rela- 
tionship between  their  effort  and  the  payoff  from  that 
effort  in  terms  of  results  during  the  last  year.  On  the 
basis  of  the  responses,  faculty  were  classified  according  to 
those  for  whom  the  payoff  was  less  than  their  effort  and 
those  for  whom  the  payoff  was  equal  to  or  greater  than  their 
effort.  The  FAL  measure  was  obtained  for  each  of  the  six 
tasks. 

A second  measure  of  leverage,  the  Composite  Assessment 
of  Leverage  (CAL),  was  constructed  From  two  other  items  on 
the  faculty  survey  which  asked  faculty  separate  questions 
about  their  perceptions  of  their  effectiveness  and  their 
perceptions  of  their  effort.  Faculty  were  placed  into  two 
response  groups  by  taking  the  ratio  of  effectiveness  to 
effort.  Those  whose  effectiveness  was  less  than  effort  were 
one  group,  and  those  whose  effectiveness  was  greater  than 
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effort  were  a second  group.  The  CAL  measure  was  calculated 
for  each  of  the  six  teaching  tasks.  Unlike  the  FAL  measure, 
the  CAL  measure  was  designed  to  allow  for  an  analysis  of 
whether  perceived  frequency  of  evaluation  had  a stronger 
relationship  to  perceived  effort  or  to  perceived  effective- 

Three  items  on  the  questionnaire  were  used  to  measure 
task  conceptions  for  each  of  the  six  tasks.  For  each  task, 
responses  were  collapsed  into  a high,  medium,  or  low  cate- 
gory reflecting  ratings  along  the  active-inert  dimension. 
The  median  response  category  was  used  to  classify  each  task 
as  high,  medium,  or  low  in  terms  of  perceived  position  along 

A summary  measure  was  also  developed  for  each  separate 
stage  of  the  evaluation  process  to  indicate  frequency  of 
activity  in  each  stage  for  a combination  of  the  six  tasks. 
Faculty  responses  to  questions  relating  to  frequency  of 
evaluation  were  collapsed  into  high,  medium,  or  low  catego- 
ries for  each  stage  of  the  evaluation  process.  A summary 
measure  of  perceived  leverage  for  a combination  of  all  tasks 
was  developed  by  asking  faculty  to  describe  their  overall 
effectiveness  in  relationship  to  effort. 

To  answer  question  number  1 of  the  problem  statement, 
gammas  were  calculated  for  each  of  the  six  teaching  tasks 
using  both  the  CAL  and  FAL  measures  of  leverage  to  determine 
whether  there  was  a significant  association  between 


perceived  frequency  of  evaluation  and  perceived  effective- 
ness in  relationship  to  effort  (leverage)  for  community 
college  faculty.  Gamma  is  a non-parametric  measure  of 
association  between  two  ordinal  variables  (Costner,  1965; 
Nie  et  al.,  1975).  Each  gamma  was  tested  for  statistical 
significance  at  the  .05  level  (Kviz  6 Knafi,  1980). 

To  answer  question  number  2 of  the  problem  statement, 
gammas  were  calculated  using  frequency  and  leverage  data 
combined  for  all  tasks  faculty  considered  high  on  the 
active-inert  dimension,  combined  for  all  tasks  faculty  con- 
sidered medium  along  the  active-inert  dimension,  and  com- 
bined for  all  tasks  faculty  considered  inert  along  the 
active-inert  dimension.  Gammas  were  compared  to  see  if  the 
relationship  between  perceived  frequency  of  evaluation  and 
perceived  leverage  was  stronger  for  tasks  conceived  as  being 

To  answer  question  number  3 of  the  problem  statement, 
the  separate  measures  of  effort  and  effectiveness  provided 
by  the  CAL  measure  were  used.  Gammas  describing  the  associ- 
ation between  perceived  frequency  and  perceived  effective- 
ness and  between  perceived  frequency  and  perceived  effort 
were  compared  to  determine  whether  perceived  effort  or  per- 
ceived effectiveness  had  a stronger  relationship  to 
perceived  frequency  for  each  task  where  there  was  a signifi- 
cant association  between  perceived  frequency  of  evaluation 
and  perceived  leverage. 
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To  answer  question  number  4 of  the  problem  statement, 
gammas  describing  the  relationship  between  perceived  fre- 
quency of  evaluation  activity  in  each  stage  of  the  evalua- 
tion process  and  perceived  leverage  were  compared  to  deter- 
mine whether  increasing  evaluation  in  a particular  stage 
might  be  more  effective  than  increasing  it  in  another  stage. 

Summary 

Background  information  and  justification  for  a study  of 
the  relationship  between  frequency  of  evaluation  and  per- 
ceived effectiveness  in  relationship  to  effort  for  a variety 
of  teaching  tasks  which  vary  along  an  active-inert  continuum 
have  been  presented  in  Chapter  I.  An  overview  of  the 
literature,  including  the  evolution  of  faculty  evaluation, 
expectancy  motivation  theory,  intrinsic  motivation,  the 
theory  of  evaluation  and  the  exercise  of  authority,  and  the 
relationship  of  other  contingency  models  of  leadership  to 
the  concept  of  task  conceptions  will  be  presented  in  Chapter 
II.  The  development  and  testing  of  the  survey  instrument, 
the  sampling  design,  and  the  procedures  for  collecting  and 
analyzing  the  data  will  be  described  in  Chapter  III.  The 
research  findings  will  be  presented  and  discussed  in  Chapter 
IV,  and  a summary  of  the  study,  conclusions,  analysis, 
recommendations,  and  implications  will  be  presented  in 
Chapter  V. 


CHAPTER  II 


REVIEW  OF  RELATED  LITERATURE 

In  reviewing  the  literature  in  the  area  of  faculty 
evaluation,  the  dominant  theme  that  emerges  is  that  faculty 
evaluation  is  in  a state  of  growth  and  continuing  change.  It 
is  a science  that  is  still  in  its  infancy  and  will 
experience  such  dramatic  changes  in  the  next  decade  that  it 
may  be  hardly  recognizable  in  its  present  form  by  the  year 
2000  (Seldin,  1984).  As  Arthur  Chickering  (1984)  noted,  "The 
days  of  benign  neglect  and  seat-of-the-pants  evaluation  are 
fast  disappearing"  (p.  91). 

In  this  chapter,  the  literature  relating  to  the 
evolution  of  faculty  evaluation  systems  up  to  the  current 
time  is  reviewed  with  emphasis  on  the  new  trends  which  are 
affecting  faculty  evaluation  in  colleges.  The  effects  of 
evaluation  on  motivation  are  discussed.  This  includes  a 
review  of  expectancy  motivation  theory  as  it  relates  to  the 
concept  of  leverage  and  a review  of  the  effects  of  evaluation 
on  intrinsic  motivation.  The  Dornbusch  and  Scott  (197S) 
theory  of  evaluation  and  the  exercise  of  authority  is 
reviewed  in  terms  of  their  research  relating  to  education 
and  in  terms  of  their  concept  of  task  conceptions  in 
relationship  to  other  contingency  theories. 
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The  Evolution  of  Faculty  Evaluation 

The  changes  in  faculty  evaluation  closely  parallel  the 
changing  conditions  in  higher  education.  Since  1970, 
greater  care  and  systematization  have  been  evident  in  evalu- 
ation practices  and  procedures.  Economic  and  political 
pressures  have  increased  the  demand  for  accountability. 
Changes  in  faculty  governance  in  the  forms  of  increased 
unionization  and  shared  control  with  nonacademic  administra- 
tors have  increased  the  demands  from  faculty  for  more 
visible  and  equitable  evaluation  systems.  The  expansion  of 
the  size  of  institutions  and  the  number  of  faculty  in  each 
institution  has  made  the  traditional  personal  and  collegial 
evaluation  procedures  obsolete  (Eble,  1976). 

Geis  (1984)  likened  the  changes  that  have  taken  place 
in  higher  education  that  relate  to  evaluation  as  growth  from 
"monastery  to  industry"  (p.  101).  Until  the  eighteenth 
century,  students  hired  and  dismissed  their  own  teachers  and 
paid  greater  fees  to  the  more  esteemed  teachers  (Seldin, 
1980;  Werdell,  1967).  Practices  similar  to  this  are  still 
seen  today  in  some  tutorial  disciplines  such  as  music  and  in 
voucher  plans  for  educational  finance  (Doyle,  1982). 

Even  as  the  traditional  organization  of  colleges  and 
universities  that  we  know  today  began  to  emerge  in  the 
eighteenth  and  nineteenth  centuries,  Geis  (1984)  said  they 
still  resembled  a monastery  in  terms  of  faculty  evaluation. 
He  described  the  faculty  as 
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The  emergence  of  the  egalitarian  philosophies  of  the 
sixties  brought  another  surge  in  growth  in  colleges  and 
universities  and  spurred  the  beginning  of  community  college 
systems  in  many  states.  With  this  change  in  philosophy  came 
the  consumerism  movement  and  the  tremendous  pressures  of 
student  unrest  (Miller,  1975). 

The  result  of  these  changes  was  that  higher  education 
became  an  industry  and  the  life  of  the  professor  became  a 
job  (Geis,  1984).  Vigorous  activity  was  evident  in  the  area 
of  evaluation  as  colleges  and  universities  struggled  to 
replace  the  controls  of  old  traditions  and  values  with  new 
systems  and  procedures. 

With  the  seventies,  the  new  problem  of  declining 
financial  resources  came  to  higher  education,  and,  as  might 
be  expected,  the  evaluation  impetus  grew  stronger.  During 
the  seventies,  entire  books  began  to  appear  on  instructional 
evaluation  (Bolton,  1973;  Centra,  1981;  Costin,  Greenbough, 
6 Menges,  1971;  Kulik  6 McKeachie,  1975;  Miller,  1972; 
Miller,  1975).  The  science  of  faculty  evaluation  had 
changed  from  small  amounts  of  fragmented  research  into  a new 
discipline  (Doyle,  1982). 

The  majority  of  evaluation  research  up  until  the 
eighties  was  centered  on  the  mechanics  of  evaluation: 
student  ratings  vs.  classroom  observation  vs.  self- 
evaluation,  the  validity  and  reliability  of  various 
evaluation  methods,  the  use  of  student  learning  to  evaluate 
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faculty  performance,  and  the  formative  vs.  summative  uses  of 
evaluation  information.  However,  in  the  eighties,  higher 
education  was  faced  with  yet  a new  problem.  Drastically 
declining  enrollments  in  many  institutions  resulted  in 
little  opportunity  to  hire  new  faculty  and  left  colleges 
with  an  aging  faculty,  hired  during  the  expansion  of  the 
sixties,  faced  with  little  mobility,  reduced  funds  for 
research,  and  less  opportunity  to  choose  the  courses  they 
taught  (Bess,  1982). 

A 1972  address  by  E.  F.  Cheit  to  the  National  Associa- 
tion of  State  Universities  and  Land  Grant  Colleges  described 
the  scene  which  was  beginning  to  emerge  even  as  early  as  the 
seventies. 

The  problem  of  how  to  bring  new  faculty  members  in  is 
becoming  the  problem  of  how  to  counsel  old  ones  out. 
Those  happy  recruitment  parties  at  the  scholarly  con- 
ventions are  being  replaced  by  dreary  technical 
meetings  on  the  actuarial  foundations  of  early  retire- 
ment. (Miller,  1975,  p.  5) 

With  these  changes,  faculty  evaluation  has  quickly 
taken  on  a new  dimension.  Attention  has  turned  from  the 
mechanics  of  evaluation,  which  occupied  so  much  of  the  time 
of  researchers  and  theorists  in  the  sixties  and  seventies, 
to  the  new  problem  of  how  to  use  evaluation  as  a way  to 
motivate  faculty  and  to  increase  faculty  satisfaction  in  a 
profession  that  is  not  as  promising  as  it  once  was. 

Motivation 

The  level  of  theoretical  activity  involving  motivation 
in  work  organizations  has  increased  during  the  last  few 
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years.  Administrators  have  identified  understanding  the 
individual  in  the  work  organization  as  their  most  difficult 
problem  (Miskel,  DeFrain,  8 Wilcox,  1980).  luthans  and 
Kreitner  (1975)  attributed  this  attention  to  motivation  to 
the  increase  in  human  problems  such  as  absenteeism,  mili- 
tancy, and  lack  of  commitment  to  work.  Steers  and  Porter 
(1975)  suggested  that  there  is  a new  recognition  that 
behavioral  factors  are  of  critical  importance  if  organiza- 
tions are  to  attract  and  retain  dependable  and  creative  per- 
formers and  that  human  resources  are  long-term  assets.  They 
further  suggested  that  pressure  from  increased  competition 
and  pressure  from  external  agencies  have  made  it  necessary 
to  seek  new  mechanisms  to  increase  effectiveness  and  effi- 

Several  hypotheses,  theories,  and  approaches  to 
organizational  effectiveness  have  resulted  from  the  study  of 
organizational  behavior.  What  makes  some  organizations  work 
more  efficiently  or  become  more  productive  than  others? 
More  specifically,  what  factors  are  involved  in  making  indi- 
viduals in  an  organization  more  efficient  or  more  productive 
than  others  within  or  outside  of  the  organization? 

Two  topics  relating  to  motivation  in  organizations  are 
closely  linked  to  the  study  of  the  effects  of  evaluation  on 
the  effort  and  effectiveness  of  community  college  faculty. 
The  first,  expectancy  motivation  theory,  has  been  a dominant 
approach  to  research  on  motivation  for  many  years  (Campbell 
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to  things  that  are  highly  valued,  and  they  work  hard  if  they 
believe  that  effort  leads  to  these  highly  valued  things. 

Galbraith  and  Cummings  (1967)  developed  a model  to 
explain  the  interrelationships  of  expectancy,  valence,  and 
instrumentality.  Expectancy  is  defined  as  the  relationship 
between  behavior  and  performance  levels  and  is  high,  for 
example,  if  a faculty  member  believes  that  it  is  highly 
probable  that  intensive  efforts  in  lecture  preparation  will 
lead  to  increased  student  achievement  and  positive  student 
attitudes  (Miskel,  DeFrain,  6 Wilcox,  1980). 

Valence  is  defined  as  the  importance  an  individual 
attaches  to  a reward  that  is  received  from  working  in  an 
organization  and  may  include  recognition,  salary,  personal 
satisfaction,  and  other  rewards  (Mowday,  1982). 

Instrumentality  is  defined  as  the  perceived  probability 
that  a reward  will  follow  from  a particular  performance 
(Mowday,  1982).  For  example,  instrumentality  is  high  if  a 
faculty  member  perceives  that  high  student  achievement  or 
positive  student  attitudes  are  likely  to  result  in  increased 
recognition  of  the  ability  of  the  faculty  member  (Miskel, 
DeFrain,  S Wilcox,  1980). 

The  basic  assumption  of  expectancy  motivation  theory  is 
that  individuals  make  conscious  decisions  on  how  to  behave 
and  that  these  decisions  are  influenced  by  evaluating  proba- 
bilities and  the  importance  of  certain  outcomes.  In 
expectancy  theory,  the  force  of  motivation  is  the  product  of 
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expectancy,  valence,  and  instrumentality  (Miskel,  DeFrain,  § 
Wilcox,  1980).  The  relationship  between  these  components 
and  examples  of  each  is  presented  in  an  adaptation  of  the 
Galbraith  and  Cummings  (1967)  model  in  Figure  1. 

Heneman  and  Schwab  (1978)  and  Mitchell  (1974)  in 
comprehensive  reviews  of  research  on  expectancy  motivation 
theory  found  that  the  force  of  motivation  is  positively 
correlated  with  job  satisfaction  and  performance.  However, 
in  most  studies,  large  amounts  of  the  variance  remained 
unexplained,  and  in  a few  studies,  contradictory  results 
were  reported  (Miskel,  Oefrain,  6 Wilcox,  1980). 

Although  expectancy,  instrumentality,  and  valence  have 
been  recognized  in  most  studies  as  interrelated  components 
in  the  determination  of  the  force  of  motivation,  much 
greater  attention  has  been  given  to  the  instrumentality  and 
valence  components  than  to  the  expectancy  component. 
Neither  Dornbusch  and  Scott  (197S)  nor  Natriello  (1984) 
mentioned  expectancy  motivation  theory,  but  the  constructs 
used  in  their  research  are  clearly  related  to  expectancy 
motivation  theory. 

Natriello's  concept  of  leverage,  which  is  expressed  as 
the  relationship  between  perceived  effort  and  perceived 
effectiveness,  is  closely  related  to  expectancy,  which  has 
been  defined  as  the  subjective  probability  that  effort  will 
lead  to  the  desired  performance  outcome  (Mowday,  1982). 
Since  expectancy  is  equal  in  terms  of  importance  to 


instrumentality  and  valence  in  the  determination  of  the 
force  of  motivation,  the  importance  of  leverage  in 
determining  performance  is  apparent. 

One  of  the  criticisms  of  Natriello's  (1984)  research  is 
that  he  relied  on  perceptions  rather  than  actual  data  on 
effectiveness  and  effort.  If  the  numerous  studies  (Heneman 
$ Schwab,  1978;  Mitchell,  1974)  in  which  it  was  found  that 
expectancy,  as  one  part  of  the  expectancy  motivation 
formula,  is  related  to  the  force  of  motivation  are  correct, 
then  the  perception  of  effort  in  relationship  to 
effectiveness  may  be  a determinant  of  performance, 
regardless  of  whether  those  perceptions  are  accurate 
reflections  of  actual  effort  and  effectiveness.  Therefore, 
any  supervisory  behavior,  including  evaluation,  that  is 
related  to  increases  in  the  perception  of  leverage  may  be 
important  in  determining  organizational  effectiveness. 

Galbraith  and  Cummings  (1967)  also  described  another 
part  of  the  expectancy  motivation  theory  that  is  related  to 
the  study  of  the  effects  of  frequency  of  evaluation  on 
faculty  leverage.  One  of  the  examples  of  valence  that 
Galbraith  and  Cummings  (1967)  used  was  recognition  of 
effort.  Evaluation  is  clearly  a mechanism  by  which  a 
department  chairman  can  provide  recognition.  However,  the 
relationship  between  evaluation  and  the  components  of  the 
expectancy  motivation  theory  becomes  stronger  if  considered 
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in  relationship  to  path-goal  theory  which  is  an 
the  expectancy  motivation  theory. 

Path-Goal  Theory 

Path-goal  theory  is  in  large  part  an  expansion  of 
expectancy  theory  concepts  into  the  leadership  domain  (House 
§ Mitchell,  1974).  According  to  House  and  Mitchell  (1974), 
in  the  path-goal  theory  of  leadership,  leaders  are  effective 
because  of  their  impact  on  subordinates'  motivation.  They 
suggested  that  a leader  is  motivating  or  satisfying  to  the 
degree  that  the  leader's  behavior  increases  subordinate  goal 
attainment  and  clarifies  the  paths  to  these  goals. 

One  of  the  first  groups  to  introduce  these  theories 
under  the  title  of  path-goal  theory  was  Georgopoulos, 
Mahoney,  and  Jones  (1957).  They  accepted  the  notion  that 
individuals  have  certain  goals  and  that  by  achieving  these 
goals,  needs  of  the  individual  are  met.  Further,  this  goal- 
directed  behavior  is  a result  of  rational  decision  making  on 
the  individual's  part.  They  postulated  that  motivation 
depends  on  the  particular  needs  of  the  individual  as 
reflected  in  the  goals  towards  which  he  is  moving  and  his 
perception  regarding  the  relative  usefulness  of  productive 
behavior  as  an  instrumentality  or  as  a path  to  attainment  of 
these  goals.  They  also  identified  freedom  from  barriers  to 
follow  the  desired  path  as  another  factor  that  affects  pro- 
ductive behavior  (Georgopoulos,  Mahoney,  $ Jones,  1957). 


Although  this  first  path-goal  theory  formulation  con- 
tained many  of  the  same  ideas  as  the  expectancy  motivation 
theory,  the  concept  of  freedom  from  barriers  to  follow  a 
chosen  path  was  added  and  thus  set  the  stage  for  later 
studies  which  added  the  new  variable  of  leadership  behavior 
as  a determinant  of  the  force  of  motivation. 

Two  other  individuals  are  acknowledged  as  the  major, 
and  more  contemporary,  contributors  to  the  path-goal  theory 
of  leadership.  Martin  Evans  (1974)  said  that  the  motiva- 
tional process  starts  with  the  supervisor  behaving  in  a cer- 
tain way.  Evans  defined  two  types  of  behavior.  The  first, 
labeled  consideration,  involves  indications  of  trust, 
respect,  concern  for  personal  needs,  two-way  communication, 
and  participation  in  decision  making.  The  second  type  of 
behavior  is  labeled  initiation  of  structure  and  focuses  on 
organizing  and  defining  work,  establishing  the  roles  of  the 
subordinate,  and  striving  for  the  desired  performance. 

These  two  forms  of  behavior  affect  the  subordinate  by 
influencing  the  perception  of  path-goal  instrumentality  (the 
extent  to  which  following  a certain  path  or  behaving  in  a 
certain  way  is  seen  as  leading  to  goal  attainment).  How- 
ever, as  Evans  (1974)  pointed  out,  motivation  is  only  one 
contributor  to  actual  job  performance.  Environmental 
factors,  the  nature  of  the  task,  and  individual  abilities 


also  important  factors. 
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This  position  agrees  with  that  of  Cummings  and  Schwab 
(1973)  who  suggested  that  there  are  three  basic  factors  that 
influence  performance  in  organizations:  ability,  motiva- 
tion, and  climate. 

In  the  path-goal  approach,  leadership  behavior  is  con- 
sidered an  independent  variable  having  a direct  effect  on 
specific  psychological  states  of  the  subordinate  (House  § 
Dessler,  1974).  House  and  Oessler  (1974)  listed  intrinsic 
job  satisfaction,  satisfaction  with  extrinsic  rewards,  the 
expectancy  that  effort  leads  to  effective  performance,  and 
the  expectancy  that  performance  leads  to  rewards  as  some  of 
the  psychological  states  of  subordinates  that  may  be 
affected  by  supervisory  behavior. 

Evans  (1974)  extended  his  theory  to  include  three  types 
of  moderator  variables  in  the  relationship  between  super- 
visor behavior  and  subordinates'  perceived  path-goal  instru- 
mentalities. These  moderator  variables  are  the  degree  of 
upward  influence  that  a supervisor  has  in  the  organization, 
the  amount  of  role  conflict  the  subordinate  is  experiencing, 
and  the  extent  to  which  the  subordinate  is  predominantly 
internal  or  external  in  control.  He  found  that  the  greater 
the  upward  influence  of  the  supervisor,  the  more  likely  he 
could  influence  the  expectancies  of  subordinates.  Also,  the 
higher  the  role  conflict,  the  less  likely  a supervisor  is  to 
be  able  to  influence  expectancies,  and  the  more  externally 
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tions,  these  attempts  will  result  in  increased 
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increase  in  performance,  but  in  a decrease  in 
satisfaction. 
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specify  types  of  leadership  behavior  that  are  appropriate  in 
different  situations  (Filley,  House,  & Kerr,  1971). 

House  and  Dessler  (1974)  demonstrated  that  task  struc- 
ture generally  determined  whether  leader  initiation  and  con- 
sideration contributed  to  subordinate  satisfaction,  positive 
expectations,  and  role  clarity.  The  strongest  findings  con- 
cerned the  relationship  between  instrumental  (directive) 
leadership  and  subordinate  satisfaction.  It  was  found  that 
instrumental  leadership  is  negatively  correlated  with  satis- 
faction under  highly  structured  tasks  and  that  this  rela- 
tionship increased  to  a positive  correlation  as  task  struc- 
ture decreased  (House  § Dessler,  1974). 

Although  authors  such  as  House,  Mitchell,  Dessler,  and 
Evans  strongly  supported  the  path-goal  theory,  Szilagyi  and 
Sims  (1974)  and  Graen  (1969)  showed  mixed  results  in  their 

Many  of  the  findings  from  path-goal  theory  research  are 
closely  tied  to  the  Dornbusch  and  Scott  (1975)  and  Natriello 
(1984)  research  on  the  relationship  between  frequency  of 
evaluation  and  job  satisfaction  and  performance.  In  addi- 
tion, the  Dornbusch  and  Scott  (197S)  concept  of  tasks  that 
vary  along  an  active-inert  dimension  is  clearly  a variation 
on  the  path-goal  theory  proposition  that  task  structure  is  a 
moderating  variable  affecting  the  types  of  leadership 
behavior  which  are  effective  in  a particular  situation.  It 
is  apparent  that  evaluation  is  one  method  that  supervisors 
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evaluation  systems,  courses  offered  because  of  faculty 
interests  are  eliminated,  variations  in  teaching  load  are 
reduced,  and  irregular  work  hours  are  discouraged  (Bess, 
1982). 

McKeachie  (1982)  concluded  that  while  there  are  some 
positive  effects  to  increased  evaluation,  there  are  also 
some  unintended  negative  effects.  One  negative  effect  men- 
tioned is  that  evaluation  increases  anxiety  and  that  faculty 
who  become  anxious  under  the  threats  of  evaluation  are 
likely  to  be  less  creative,  more  rigid,  less  effective  in 
solving  problems,  and  to  use  less  effective  methods  of 
teaching  and  learning.  Gruenfeld  and  Weissenberg  (1966) 
found  a natural  strain  in  human  relations  as  the  result  of 
evaluation. 

The  second  negative  effect  mentioned  by  McKeachie 
(1982)  is  that  while  evaluation  may  reduce  poor  teaching 
resulting  from  inadequate  preparation  and  lack  of  attention 
to  teaching  responsibilities,  it  is  also  likely  to  result  in 
less  innovation,  less  emphasis  on  long-term  educational  out- 
comes, and  less  enthusiasm  and  commitment  to  teaching.  Link 
(1969)  found  that  "a  rating  scale  becomes  a shaping  device 
no  matter  how  supportive  the  supervisor,  the  principal,  or 
the  system"  (p.  IS).  Csikszentmihalyi  (1978)  argued  that 
this  kind  of  lack  of  enjoyment  in  teaching  causes  the  teach- 
ing activity  to  lose  its  primary  value  for  both  the  faculty 
member  and  the  student,  because  when  a teacher  teaches  only 
for  extrinsic  reasons  (pay,  high  evaluations,  etc.),  he 
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Because  of  the  many  variables  that  affect  the  path  between 
faculty  effort  and  student  gain,  many  faculty  doubt  that 
what  they  do  does  make  a difference  in  the  lives  of  their 
students  (Cohen,  1973). 

The  second  of  the  psychological  states  proposed  by 
Hackman  and  Oldman  (1980)--experienced  responsibility  for 
work  outcomes--is  also  a problem  for  faculty.  Because 
American  higher  education  is  founded  on  a strong  tradition 
of  academic  freedom,  faculty  have  generally  not  been  moni- 
tored or  held  accountable  for  their  teaching  (Doyle,  1982). 
Although  there  have  been  many  cautions  against  possible 
negative  effects  of  evaluation.  Pelt  and  Andrews  (1966) 
noted  that  complete  autonomy  without  some  constraints, 
whether  bureaucratic  or  normative,  results  in  lowered  pro- 
ductivity. Lewis  (197S)  stated  that  the  fact  that  research 
productivity,  other  service  activities,  or  longevity  are 
given  much  greater  weight  than  teaching  in  decisions  on 
salary  and  promotions  reinforces  the  perception  of  faculty 
members  that  they  have  little  accountability  for  their 
teaching  efforts.  Bess  (1982)  suggested  that  this  lack  of 
accountability  plus  the  difficulties  in  determining  whether 
teaching  efforts  are  productive  leads  to  a low  probability 
that  faculty  will  perceive  a relationship  between  their  per- 
formance and  the  desired  psychological  state  of  experienced 


responsibility  for  work 


ically 


This  lack  of 


is  a denial  of 


Suffice  it  to  say  that  the  ambiguity  of  both  formal  and 

into  roles  that  provide  little  intrinsic’  reward, 

11:^11 


(1982)  called  intrinsic 


44 

solutions  to  the  common  difficulties  in  teaching  and 
observed  a pattern  of  defensiveness,  self-protectionism,  and 
a reduction  in  the  importance  attached  to  teaching. 

Morgan  (1984)  summarized  a plethora  of  research  studies 
on  the  conditions  which  are  likely  to  diminish  and  enhance 
intrinsic  motivation.  In  some  of  this  research,  rewards  of 
all  kinds  were  found  to  decrease  intrinsic  motivation.  In 
other  research,  competition  (Deci  8 Ryan,  1980),  surveil- 
lance (Lepper  8 Green,  1975),  and  evaluation  of  performance 
(Smith,  1974)  were  also  found  to  decrease  intrinsic  motiva- 

Deci  and  Ryan  (1980)  discussed  a theory  called  cogni- 
tive evaluation  theory  and,  based  on  this  theory,  stated 
that  rewards,  surveillance,  competition,  and  evaluation 
change  the  perceived  locus  of  causality  from  internal  to 
external  and  leave  a person  feeling  less  self-determining. 
When  an  activity  is  engaged  in  to  get  a reward  or  to  comply 
with  a constraint  the  activity  becomes  instrumental  and  is 
no  longer  engaged  in  for  its  own  sake  (Deci  8 Ryan,  1982). 

Zuckerman  et  al.  (1978)  found  that  when  people  are 
given  a choice  about  which  task  to  engage  in  or  how  to  pro- 
ceed on  a task,  they  are  more  intrinsically  motivated. 

Deci  and  Ryan  (1980)  also  found  that  positive  feedback 
leads  to  greater  intrinsic  motivation  and  that  failure  and 
negative  feedback  tend  to  decrease  motivation.  This  is  tied 
to  the  second  part  of  cognitive  evaluation  theory  which  Deci 


and  Ryan  (1980)  interpreted  as  meaning  that  intrinsic  moti- 
vation is  tied  to  perceived  competence. 

Other  researchers  have  begun  to  find  evidence  that  con- 
tradicts the  idea  that  rewards  decrease  intrinsic  moti- 
vation. Deci  (1975)  found  that  rewards  have  both  a control- 
ling and  an  informational  component.  He  hypothesized  that 
to  the  extent  an  activity  is  seen  as  a means  to  an  end,  in 
other  words  is  instrumental  in  receiving  a reward,  the  task 
will  be  seen  as  less  inherently  interesting  and  will  be  less 
likely  to  be  engaged  in  if  rewards  are  absent.  If,  on  the 
other  hand,  rewards  convey  information  on  a person's  compe- 
tence on  a task,  intrinsic  liking  for  the  task  will  increase 
and  the  task  will  be  more  likely  to  be  engaged  in  if  rewards 
are  absent.  This  research  has  been  supported  in  studies  by 
Enzle  and  Ross  (1978)  and  Ryan  (1982). 

Deci  and  Ryan  (1982),  commenting  on  the  conditions 
which  motivate  faculty,  wrote: 

Times  are  becoming  more  difficult  in  colleges  and 
universities,  and  the  pressure  appears  to  be  building. 
Thus,  it  is  increasingly  imperative  that  administrators 
work  to  create  the  conditions  that  promote  rather  than 
drain  intrinsic  motivation.  The  keys  to  creating  these 
conditions  seem  to  be  administrators'  orientations 
toward  autonomy  and  structures  that  inform  rather  than 
control.  If  conditions  are  created  that  bolster 
teachers'  feelings  of  competence  and  self-determina- 
tion, teachers'  motivation  to  teach  will  take  care  of 
itself,  (p.  34) 

Mihaly  Csikszentmihalyi  (1978)  studied  intrinsically 
motivated  individuals  and  found  that  regardless  of  the 
activity  involved,  intrinsically  motivated  persons  regularly 


The  Theory  of  Evaluation  and  the 


Dornbusch  and  Scott  (1975)  developed  the  theory  of 
evaluation  and  the  exercise  of  authority  after  a decade-long 
research  program  in  a wide  variety  of  organizations.  There 
are  18  propositions  with  numerous  theorems  and  corollaries 
relating  to  power  and  authority  in  organizations  in  the 
complete  theory.  The  parts  of  the  theory  and  research  that 
relate  either  to  higher  education  or  to  the  effects  of 
frequency  of  evaluation  are  considered  in  this  review. 

Frequency  of  Evaluation 

Using  workers  in  five  organizations,  Dornbusch  and 
Scott  (1975)  tested  the  hypothesis  that  frequency  of  evalua- 
tion is  negatively  correlated  with  the  organizational  status 
of  the  respondent.  They  found  no  relationship  between 
reported  frequency  of  evaluation  and  position  in  the  organi- 
zation. Teachers  included  in  this  study  reported  a median 
frequency  of  evaluation  of  "seldom"  for  all  four  teaching 

In  this  same  study,  they  found  that  frequency  of  evalu- 
ation was  positively  correlated  with  satisfaction  with  the 
evaluation  process  (Dornbusch  5 Scott,  1975). 

In  another  study,  they  again  tested  the  relationship 
between  frequency  of  evaluation  and  satisfaction  using  131 
teachers  from  six  schools  in  a single  school  district.  For 
each  of  the  four  tasks  performed  by  teachers,  they  found  a 
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strong  positive  relationship  between  frequency  of  evaluation 
and  satisfaction  with  the  manner  in  which  tasks  were 
assigned  and  evaluated  (Dornbusch  8 Scott,  1975). 

When  the  same  study  was  replicated  using  teachers  in 
alternative  schools,  they  found  only  a slightly  positive 
correlation  between  frequency  of  evaluation  and  satisfaction 
(Dornbusch  8 Scott,  1975). 

In  another  part  of  their  research  in  the  six  schools, 
Dornbusch  8 Scott  (1975)  studied  the  differences  in  the  per- 
ceptions of  teachers  and  principals  as  to  the  frequency  and 
content  of  evaluations.  They  found  that  for  all  four  teach- 
ing tasks,  principals  believed  themselves  to  be  communi- 
cating evaluations  more  frequently  than  was  reported  by 
teachers.  Principals  also  reported  communicating  negative 
evaluations  far  more  often  than  teachers  reported  receiving 
them.  Dornbusch  and  Scott  (1975)  considered  this  to  be 
important  because  of  the  earlier  research  results  in  which 
they  found  a relationship  between  frequency  of  evaluation 
and  satisfaction.  They  felt  that  because  of  the  combined 
findings  of  the  two  studies,  it  is  very  important  that  a 
superior  recognize  the  possibility  of  discrepant  perceptions 
of  frequency  of  evaluation,  and  particularly  of  communicated 
negative  evaluation. 

By  comparing  teachers  in  open-space  classrooms  with 
teachers  in  a traditional,  one-teacher  classroom,  Dornbusch 
and  Scott  (197S)  studied  the  relationship  between  the 
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visibility  of  work  and  a performer's  satisfaction  with 
evaluations.  They  found  a strongly  positive  correlation 
between  perceived  visibility  of  work  and  satisfaction  with 
evaluation.  This  concept  of  visibility  is  closely  related 
to  the  sampling  step  in  the  six-stage  model  of  the 
evaluation  process. 

In  another  study  using  data  obtained  from  131  teachers, 
the  Dornbusch  and  Scott  (1975)  team  analyzed  the  relation- 
ship between  the  power  of  subordinates  and  satisfaction  with 
evaluation.  The  hypothesis  that  subordinates  who  believed 
they  could  influence  the  selection  of  criteria  were  more 
satisfied  with  the  criteria  and  with  evaluation  in  general 
was  overwhelmingly  supported. 

Research  in  Universities 

Using  data  collected  in  a large,  private  research  uni- 
versity, Robert  Hind  (1971),  under  the  supervision  of 
Dornbusch  and  Scott,  investigated  authority  relationships 
and  evaluation  processes  in  a university  faculty.  This 
research  was  the  first  attempt  to  study  variations  in 
authority  systems  as  a function  of  the  type  of  task  being 
performed.  Hind  (1971)  selected  four  tasks  which  typically 
are  performed  by  faculty:  classroom  teaching,  research  and 
scholarship,  university  service,  and  service  external  to  the 
university. 
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along  a continuum  from  active  to  inert  and  the 
characteristics  described  by  Fiedler  as  varying  from  low  in 
control  to  high  in  control.  Although  Dornbusch  and  Scott 
focused  on  differences  in  tasks  that  one  individual  performs 
and  Fiedler  (1967)  focused  on  the  characteristics  of  dif- 
ferent jobs  within  an  organisation,  both  share  the  commonal- 
ity that  different  supervisory  behaviors  are  effective  in 
different  situations.  The  task  structure  component  used  by 
Fiedler  as  part  of  his  definition  of  low  and  high  control  is 
almost  identical  to  the  definition  of  task  structure  used  by 
Dornbusch  and  Scott.  It  is  difficult  to  directly  compare 
the  two  theories,  because  Fiedler  did  not  include  evaluation 
as  a component  of  his  definition  of  leader  behavior,  but 
similarities  can  be  seen  in  the  Fiedler  conclusion  that  low 
control  situations  and  high  task-oriented  leaders  are  most 
compatible  and  the  Dornbusch  and  Scott  finding  that  fre- 
quency of  evaluation  is  more  strongly  related  to  satisfac- 
tion and  leverage  for  tasks  that  are  more  active. 

A second  contingency  approach  to  leadership- -path-goal 
theory--has  already  been  reviewed.  Contrary  to  Fiedler's 
model,  it  is  assumed  in  path-goal  theory  that  the  leadership 
style  of  an  individual  can  vary  as  situations  within  the 
organization  vary  due  to  personal  characteristics  of  subor- 
dinates and  due  to  environmental  factors  which  stimulate, 
constrain,  or  reward  the  motivation  of  workers.  A leader- 
ship style  will  be  effective  in  a path-goal  sense  if  it 
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Vroora  and  Yetton  (1973)  identified  these  five  decision- 
making styles,  each  requiring  a different  degree  of  subordi- 
nate participation: 

1.  Leader  makes  the  decisions  using  whatever  informa- 
tion is  available. 

2.  Leader  secures  necessary  information  from  members 
of  the  group,  then  makes  the  decision. 

3.  Leader  shares  the  problem  with  relevant  members  of 
the  group  on  a one-to-one  basis,  then  makes  the  decision. 

4.  Leader  shares  the  problem  with  members  as  a group 
at  a meeting,  then  decides. 

5.  Leader,  acting  as  a chairperson,  shares  the 
problem  with  the  group  and  facilitates  efforts  of  the  group 
to  reach  consensus  on  a group  decision. 

By  answering  a series  of  questions,  the  proper  deci- 
sion-making style  can  be  determined  for  each  of  14  different 
decision  situations.  The  questions  relate  to  the  necessity 
for  a quality  decision,  the  likelihood  and  necessity  of 
acceptance  by  the  subordinate,  and  agreement  on  organiza- 
tional goals  (Kreitner,  1986). 

Although  decision  making  and  evaluation  may  not  seem  to 
be  related,  a review  of  the  Dornbusch  and  Scott  (1975)  and 
Natriello  (1984)  six-step  model  of  the  evaluation  process 
reveals  a close  interconnection.  The  first  three  stages  of 
the  evaluation  model--allocating,  criteria-setting,  and 
sampling--involve  significant  decisions  as  to  how  work  will 
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be  done.  If  the  seven  Vroom  and  Vetton  questions  (1973)  are 
answered  for  different  teaching  tasks,  different  appropriate 
decision-making  styles  are  indicated. 

This  is  closely  related  to  the  Dornbusch  and  Scott 
(1975)  concept  of  task  conceptions  in  which  they  stated  that 
subordinates  desire  different  work  and  supervisory  arrange- 
ments for  different  tasks.  Natriello  (1984)  hypothesized 
that  frequency  of  evaluation  is  more  strongly  related  to 
increased  perceptions  of  effectiveness  in  relationship  to 
effort  for  tasks  that  are  more  active.  This  can  be  readily 
related  to  the  Vroom  and  Yetton  proposition  that  structured 
and  unstructured  tasks  require  different  levels  of  subordi- 
nate participation  in  the  decision-making  process. 

Another  contingency  theory  of  leadership  that  has 
attracted  considerable  attention  is  what  Paul  Hersey  and 
Kenneth  Blanchard  (1977)  called  life-cycle  theory  or  situa- 
tional leadership  theory.  The  theory  is  based  on  leadership 
studies  known  as  the  Ohio  State  Studies  and  on  the  work  of 
William  Reddin  (Megginson,  Mosley,  S Pietri,  1986). 

The  basic  concept  of  this  theory  is  that  a leader's 
behavior  should  be  situational,  based  primarily  on  the 
maturity  or  immaturity  of  followers  (Megginson,  Mosley,  5 
Pietri,  1986).  Hersey  and  Blanchard  (1977)  defined  maturity 
as  the  capacity  of  individuals  to  set  high,  realistic  goals 
and  as  their  willingness  and  ability  to  take  responsibility 


for  a specific  task. 


Hersey  and  Blanchard  (1977)  proposed  that  as  the 
maturity  level  of  subordinates  increases,  the  most  effective 
leadership  style  changes  from  high  task-low  relationship  to 
moderate  task-high  relationship  and,  finally,  to  low  task- 
low  relationship.  They  defined  task  behavior  as  the  extent 
to  which  leaders  organize  and  define  the  roles  of  their  fol- 
lowers and  tell  them  when,  where,  and  how  tasks  are  to  be 
accomplished.  They  defined  relationship  behavior  as  deter- 
mined by  the  amount  of  support  provided  by  the  leader  and 
the  extent  to  which  the  leader  engages  in  interpersonal  com- 
munication and  facilitating  behavior  (Megginson,  Mosley,  5 
Pietri,  1986). 

If  faculty  are  considered  to  be  highly  mature  individ- 
uals, a low  relationship-low  task  leadership  style  is  the 
most  effective,  according  to  the  Hersey  and  Blanchard  (1977) 
model.  This  seems  to  be  directly  contrary  to  the  research 
by  Dornbusch  and  Scott  (197S)  and  Natriello  (1984)  in  which 
they  found  a positive  relationship  between  frequency  of 
evaluation  and  satisfaction  and  effectiveness  in 
relationship  to  effort.  Perhaps  the  two  can  be  reconciled 
by  considering  the  possibility  that  the  pressures  of  declin- 
ing enrollments,  decreasing  mobility,  less  well-prepared 
students,  and  increased  demands  for  accountability  may  have 
so  weakened  both  the  willingness  and  perceived  ability  of 
faculty  that  they  can  no  longer  be  considered  as  at  the 


highest  level  of  maturity,  using  the  Hersey  and  Blanchard 
(1977)  definition. 

Summary 

In  this  chapter,  selected  literature  related  to  the 
effects  of  evaluation  on  faculty  has  been  reviewed.  The 

evolution  of  faculty  evaluation  has  shown  a movement  from 
virtually  no  systematized  evaluation  prior  to  World  War  II 
to  a tremendous  interest  in  the  mechanics  of  evaluation  in 
the  sixties  and  seventies  to  increased  attention  to  the 
effects  of  evaluation  on  faculty  in  the  eighties. 

Although  Dornbusch  and  Scott  (1975)  in  their  research 
at  Stanford  were  the  first  formally  to  study  the  relation- 
ship between  evaluation  and  faculty  satisfaction  and  to  pro- 
pose the  idea  of  task  conceptions  as  determinants  of  appro- 
priate supervisory  behavior,  it  is  evident  from  the  review 
of  literature  in  this  chapter  that  the  constructs  of  the 
Dornbusch  and  Scott  theory  are  closely  related  to  those  in 
expectancy  motivation  theory,  path-goal  theory,  intrinsic 
motivational  theory,  and  other  contingency  theories. 

The  concept  of  leverage,  developed  by  Natriello  (1984) 
as  an  extension  of  the  Dornbusch  and  Scott  (1975)  research, 
was  found  to  be  closely  related  to  the  expectancy  portion  of 
the  formula  in  expectancy  motivation  theory  in  which  a 
person's  perception  that  their  efforts  lead  to  desired  per- 
formance is  held  to  be  a determinant  of  motivation  to  per- 
The  concepts  of  expectancy  (and 
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leverage)  are  further  extended  into  the  leadership  domain  by 
the  path-goal  theory  postulate  that  leadership  behavior  is 
effective  to  the  extent  that  it  increases  a subordinate's 
perception  that  their  efforts  will  lead  to  desired  perform- 
ance and  desired  rewards.  Thus,  a theoretical  basis  for 
Natriello's  hypothesis  that  increased  frequency  of  evalua- 
tion is  related  to  increased  perceptions  of  effectiveness  in 
relationship  to  effort  is  provided  by  path-goal  theory. 

While  there  is  disagreement  among  researchers  as  to  the 
effects  of  evaluation  on  the  intrinsic  motivation  of 
faculty,  substantial  evidence  has  been  provided  that  evalua- 
tion may  be  an  important  factor  in  increasing  the  intrinsic 
satisfaction  of  faculty  by  increasing  feelings  of  compe- 
tence, responsibility,  and  control. 

The  Dornbusch  and  Scott  (1975)  idea  that  different 
tasks  require  different  supervisory  behaviors  to  maximize 
effectiveness  is  supported  by  the  contingency  formulations 
of  leadership.  Dornbusch  and  Scott  went  one  step  beyond  the 
other  contingency  theories  by  breaking  down  the  work  of  a 
single  individual  into  different  tasks  requiring  different 
leadership  behaviors  for  maximum  effectiveness. 

In  studies  conducted  in  public  schools,  Dornbusch  and 
Scott  (1975)  and  Natriello  (1984)  have  provided  empirical 
evidence  that  frequency  of  evaluation  is  related  to  satis- 
faction and  leverage.  This  relationship  is  firmly  supported 
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by  the  several  theories  which  have  been  reviewed  in  this 

This  project  used  constructs  and  techniques  similar  to 
those  of  Dornbusch  and  Scott  (197S)  and  Natriello  ( 1984)  to 
gather  additional  empirical  evidence  that  this  relationship 
is  true  for  community  college  faculty. 


CHAPTER  III 


METHODS  AND  PROCEDURES 
The  Problem 

The  purpose  of  this  study  was  to  examine  the  relation* 
ship  between  the  perceptions  of  community  college  faculty  of 
the  frequency  of  evaluation  activity  and  their  perceptions 
of  their  own  effectiveness  in  relationship  to  their  effort 
on  these  six  teaching  tasks:  classroom  teaching,  curriculum 
development,  professional  development,  college  service, 
student  advising,  and  maintenance  activities. 

Specifically,  the  following  questions  were  addressed: 

1.  For  each  of  the  six  teaching  tasks,  was  there  a 
statistically  significant  association  (j)  < .OS)  between  per- 
ceived frequency  of  evaluation  and  perceived  effectiveness 
in  relationship  to  effort  (leverage)? 

2.  Was  there  a stronger  relationship  between  per- 
ceived frequency  of  evaluation  and  leverage  for  tasks  that 
faculty  perceived  as  active,  for  tasks  faculty  perceived  as 
moderately  active,  or  for  tasks  faculty  perceived  as  inert? 

3.  For  each  of  the  six  teaching  tasks  where  there  was 
a statistically  significant  association  (£  < .05)  between 
perceived  frequency  of  evaluation  and  leverage,  was 


perceived  frequency  of  evaluation  related  to  increases  in 
perceived  effectiveness,  decreases  in  perceived  effort,  or 

4.  In  which  stages  of  the  evaluation  process  did  per- 
ceived frequency  of  evaluation  have  the  strongest  associa- 
tion with  perceived  effectiveness  in  relationship  to  effort 
(leverage)  for  a combination  of  the  six  teaching  tasks? 

Questionnaire  Design 

A survey  research  design  was  adopted  for  this  project, 
and  a questionnaire  was  developed  to  determine  the  relation- 
ship between  perceived  frequency  of  evaluation  and  perceived 
effectiveness  in  relationship  to  effort  for  a variety  of 
teaching  tasks.  The  research  design  was  similar  to  the 
design  used  by  Natriello  (1984)  in  his  study  of  the  effects 
of  evaluation  on  elementary  school  teachers.  The  question- 
naire was  designed  to  collect  data  in  four  areas: 

1.  Perceptions  of  respondents  of  the  frequency  with 
which  they  were  evaluated  in  each  of  the  five  stages  of  the 
six-step  model  of  the  evaluation  process  (Natriello,  1984) 
for  six  different  teaching  tasks.  The  fourth  stage  of  the 
evaluation  model--appraisal--was  not  included  since  this  is 
not  visible  to  a faculty  member. 

2.  Perceptions  of  respondents  of  their  effectiveness 
in  relationship  to  effort  for  each  of  the  six  teaching  tasks 
and  for  the  combination  of  the  six  tasks. 


well  the 


teaching  task  were  included  on  the  questionnaire  (Appendix 
A). 

The  questionnaire  consisted  of  four  sections.  Section 
I contained  questions  used  to  construct  the  measure  of  fre- 
quency of  evaluation.  Faculty  were  asked  the  following 
questions: 

1.  For  each  task,  how  often  does  your  department 
chairman  (or  other  immediate  supervisor)  tell  you  what 
should  be  accomplished  (i.e.,  what  your  goals  should  be)? 

2.  For  each  task,  how  often  do  you  learn  of  the 
criteria  and  standards  that  your  department  chairman  (or 
other  immediate  supervisor)  uses  in  supervising  and  evalu- 
ating your  performance? 

3.  For  each  task,  how  frequently  do  you  think  your 
department  chairman  (or  other  immediate  supervisor)  observes 
aspects  of  your  task  performance? 

4.  How  frequently  do  you  receive  feedback  (formal  or 
informal)  on  how  well  or  poorly  you  are  doing  on  each  task? 

5.  How  frequently  does  your  department  chairman  (or 
other  immediate  supervisor)  use  the  information  from  evalua- 
tions of  your  performance  on  these  tasks  to  work  together 
with  you  to  plan  ways  to  improve  performance? 

Faculty  responded  to  the  five  questions  for  each  of  the 
six  teaching  tasks  using  Likert  scale  response  categories. 
The  scale  was  (1)  Very  Frequently,  (2)  Frequently,  (3) 


Sometimes,  (4)  Seldom,  and  (5)  Almost  Never.  The  five 
questions  corresponded  to  the  five  steps  of  the  evaluation 

Section  II  contained  questions  used  to  obtain  two 
measures  of  leverage  for  each  of  the  six  teaching  tasks. 
The  first,  the  Faculty  Assessment  of  Leverage  (FAL),  was 
determined  by  asking  faculty  this  question: 

Teaching  tasks  involve  both  some  level  of  effort  on 
your  part  and  some  level  of  payoff  in  terms  of  results. 
For  each  task,  which  of  the  following  best  describes 
the  relationship  between  your  effort  and  the  payoff 
during  this  past  year? 

Faculty  responded  by  selecting  one  of  nine  response  catego- 
ries composed  of  one  of  three  ratings  of  effort  (high, 
medium,  low)  and  one  of  three  ratings  of  payoff  (high, 
medium,  low). 

A final  question  asked  faculty  to  consider  all  of  their 
teaching  tasks  together  and  to  describe  the  relationship 
between  their  effort  and  the  payoff  in  terms  of  results 
during  the  past  year.  The  nine  response  categories  used  for 
the  FAL  measurement  for  individual  tasks  were  used. 

The  second  measure  of  leverage,  the  Composite  Assess- 
ment of  Leverage  (CAL),  was  constructed  by  asking  faculty 
these  two  questions: 

1.  Compared  to  earlier  in  your  faculty  career,  how 
effective  have  you  been  in  achieving  your  goals  for  each 


task  this  year? 
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2.  Compared  to  earlier  in  your  faculty  career,  how 
hard  are  you  working  on  each  task  this  year? 

Faculty  responded  for  each  of  the  six  tasks  using  these 
response  categories  to  describe  their  effectiveness:  (1) 

Much  Less  Effective,  (2)  Less  Effective,  (3)  About  as  Effec- 
tive, (4)  More  Effective,  and  (S)  Much  More  Effective.  They 
responded  using  these  response  categories  to  describe  their 
effort:  (1)  Not  Nearly  as  Hard,  (2)  Not  Quite  as  Hard,  (3) 

About  as  Hard,  (4)  Somewhat  Harder,  and  (5)  Much  Harder. 

Section  III  contained  questions  designed  to  measure  the 
task  conceptions  of  faculty  for  each  of  the  six  teaching 
tasks  as  varying  along  the  active-inert  continuum  described 
by  Dornbusch  and  Scott  (1975).  These  questions  were  asked 
for  each  of  the  six  tasks  to  measure  task  conception  in 
terms  of  clarity,  predictability,  and  efficacy: 

1.  How  vague  or  undefined  do  your  goals  for  each  task 
appear  to  you? 

2.  For  each  task,  how  often  can  you  predict  which  way 
of  doing  things  is  most  likely  to  enable  you  to  reach  your 

3.  For  each  task,  how  often  are  you  successful  in 
reaching  your  goals? 

The  scale  for  the  responses  to  question  1 was  (1)  Very 
Vague  and  Undefined,  (2)  Somewhat  Vague  and  Undefined,  (3) 
Occasionally  Vague  and  Undefined,  and  (4)  Almost  Never  Vague 
and  Undefined.  The  scale  for  responses  to  questions  2 
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and  3 was  (1)  Very  Frequently,  (2)  Frequently,  C3)  Some- 
tines,  and  (4)  Almost  Never. 

After  the  initial  development  of  the  questionnaire,  it 
was  administered  to  25  community  college  faculty  from  three 
colleges  that  were  not  part  of  the  sample.  Each  person  was 
interviewed  to  determine  which  questions  were  unclear,  mis- 
leading, or  anxiety-producing.  Modifications  were  made 
based  on  these  recommendations.  This  pretest  also  provided 
the  information  that  the  average  time  to  complete  the  ques- 


Sample  Selection 

The  target  population  for  this  study  was  all  full-time 
faculty  members  in  the  28  colleges  in  the  Florida  community 
college  system.  The  list  of  faculty  members  for  each 
college  was  based  on  the  listing  in  the  1985-86  college 

A multistage  cluster  sampling  technique  with  probabil- 
ity proportionate  to  size  was  used  to  limit  the  number  of 
clusters  in  the  sample  to  five,  but,  at  the  same  time,  to 
ensure  that  each  faculty  member  at  all  colleges  had  an  equal 
chance  of  selection.  The  first  step  in  this  process  was  to 
determine  the  total  number  of  full-time  faculty  at  each  col- 
lege. When  a college  had  less  than  100  faculty  members,  it 
was  combined  with  the  college  nearest  in  geographic 
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location,  and  the  resulting  combination  was  treated  as  one 
cluster  for  sampling  purposes. 

Each  cluster  was  assigned  numbers  in  proportion  to 
their  size.  For  example,  because  Miami  Dade  had  16%  of  the 
total  faculty,  they  were  assigned  16  numbers  and  because 
Palm  Beach  had  4%  of  the  total  faculty,  they  were  assigned  4 
numbers.  Five  clusters  were  then  selected  using  computer- 
generated random  numbers. 

After  the  five  clusters  were  selected,  100  full-time 
faculty  members  were  selected  from  each  college  using  compu- 
ter-generated random  numbers.  Each  college  was  then  asked 
to  verify  the  list,  and  alternates  were  randomly  selected  as 


Data  Collection  Procedures 

A cover  letter  (Appendix  B)  and  the  questionnaire 
(Appendix  A)  were  sent  to  the  500  faculty  members  selected 
for  the  sample.  A stamped  return  envelope  was  included. 
The  participants  were  requested  to  complete  the  question- 
naire and  to  return  it  in  the  enclosed  return  envelope. 

A key  administrator  was  selected  at  each  college  in  the 
five  clusters  to  assist  in  the  follow-up  of  questionnaires 
that  were  not  returned.  A letter  (Appendix  B)  was  sent  to 
each  explaining  the  procedures  and  timetable  for  follow-up. 

Three  weeks  after  the  original  questionnaires  were 
mailed,  a list  of  non-responders  was  prepared  for  each 


college.  This  was  done  from  a code 
envelope,  because  respondents  were  not  asked  to  include 
their  names  on  the  questionnaire.  This  list  was  provided 
with  additional  questionnaires  and  stamped  return  envelopes 
to  the  project  coordinator  at  each  site.  Each  coordinator 
was  asked  to  distribute  these  to  non* responders  and  to 
encourage  them  to  complete  the  questionnaire  and  to  return 
it.  At  some  institutions,  this  process  had  to  be  repeated  a 
second  time  to  meet  the  goal  of  an  801  response  rate. 

Analysis  of  the  Data 

To  develop  a measure  of  frequency  of  evaluation  for 
each  of  the  six  teaching  tasks,  respondents  were  divided 
into  three  groups.  Those  reporting  no  stage  of  the  evalua- 
tion process  in  which  they  were  evaluated  either  very  fre- 
quently or  frequently  were  classified  as  low  frequency. 
Those  reporting  one  stage  as  being  evaluated  very  frequently 
or  frequently  were  classified  as  medium  frequency,  and  those 
reporting  two  or  more  stages  as  being  evaluated  very  fre- 
quently or  frequently  were  classified  as  high  frequency. 

To  obtain  the  FAL  measure  of  leverage  for  each  of  the 
teaching  casks,  faculty  were  classified  according  to  those 
who  chose  a response  category  in  Part  A of  Section  II  in 
which  payoff  was  less  than  effort  (leverage  < 1)  and  those 
for  whom  payoff  was  equal  to  or  greater  than  effort 
(leverage  > or  = to  1). 


Responses 


questions  in  Part  B of  Section  II 


used  to  calculate  the  second  measure  of  leverage-'the  Com* 
posite  Assessment  of  Leverage  (CAL)--for  each  of  the  six 
teaching  tasks.  The  CAL  measure  was  constructed  by  taking 
the  ratio  of  effectiveness  to  effort  and  placing  faculty 
into  two  groups,  those  whose  effectiveness  was  less  than 
their  effort  and  those  whose  effectiveness  was  equal  to  or 
greater  than  their  effort. 

To  answer  question  number  1 of  the  problem  statement, 
gammas  describing  the  association  between  perceived  fre- 
quency of  evaluation  and  leverage  were  calculated  for  each 
of  the  six  teaching  tasks  using  both  the  FAL  and  CAL  measure 
of  leverage.  Each  gamma  was  tested  for  significance  by 
transferring  it  into  a normal  distribution  and  calculating  a 
£ score  using  a two-tailed  test  and  a 951  confidence  level 
(Kviz  5 Knafi,  1980). 

To  answer  question  number  2 of  the  problem  statement, 
each  of  the  six  teaching  tasks  was  classified  along  the 
active-inert  continuum  using  the  responses  to  questions  in 
Section  III  of  the  questionnaire.  If  a respondent  selected 
either  of  the  two  most  active  response  categories  (very 
vague/somewhat  vague  for  question  1,  or  sometimes/almost 
never  for  questions  2 and  3)  for  two  or  more  of  the  three 
questions,  the  task  was  classified  as  active  for  that 
respondent.  If  a respondent  selected  either  of  the  two  most 
active  response  categories  for  one  of  the  three  questions. 
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the  task  was  classified  as  moderately  active  for  that 
respondent.  If  the  respondent  did  not  select  either  of  the 
two  most  active  response  categories  for  any  of  the  three 
questions,  the  task  was  classified  as  inert  for  that 
respondent . 

The  category  which  included  the  median  for  the  respon- 
dents was  used  to  classify  each  of  the  six  tasks  as  active, 
moderately  active,  or  inert.  The  data  relating  to  perceived 
frequency  and  perceived  leverage  for  tasks  with  a similar 
classification  along  the  active-inert  continuum  were  col- 
lapsed. For  example,  since  both  classroom  teaching  and 
maintenance  factors  were  classified  as  inert  tasks,  the  data 
were  added  and  the  combination  was  treated  as  one  task. 

Gammas  describing  the  relationship  between  perceived 
frequency  of  evaluation  and  perceived  leverage  were  then 
calculated  for  the  composite  active,  moderately  active,  and 
inert  tasks.  A test  of  significance  was  performed  for  each 
gamma,  and  the  gammas  were  compared  for  strength  of  associa- 
tion to  answer  question  number  2. 

To  answer  question  number  3 and  to  determine  whether 
possible  relationships  between  perceived  frequency  of  evalu- 
ation and  perceived  leverage  were  due  to  decreases  in  per- 
ceived effort,  increases  in  perceived  effectiveness,  or 
both,  gammas  were  calculated  using  the  separate  effort  and 
effectiveness  components  that  were  part  of  the  CAL  measure 
of  leverage.  One  gamma  was  an  expression  of  the 
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relationship  between  perceived  frequency  of  evaluation  and 
perceived  effectiveness,  and  another  was  an  expression  of 
the  relationship  between  perceived  frequency  of  evaluation 
and  perceived  effort.  Both  gammas  were  tested  for  signifi- 
cance and  compared  for  strength  of  association  for  each  task 
where  there  was  a significant  association  between  perceived 
frequency  of  evaluation  and  perceived  leverage. 

To  answer  question  number  4 concerning  the  strength  of 
association  between  perceived  frequency  of  evaluation  in  any 
single  stage  of  the  evaluation  process  and  perceived 
leverage,  gammas  were  calculated  for  the  combination  of  the 
six  teaching  tasks  by  classifying  respondents  into  high, 
medium,  and  low  categories  of  perceived  frequency  of  evalua- 
tion for  each  of  the  five  stages  of  the  evaluation  process. 
Responses  were  classified  as  high  frequency  for  a given 
stage  of  the  evaluation  model  if  the  respondent  indicated 
very  frequently  or  frequently  for  that  stage  for  at  least 
three  of  the  six  teaching  tasks.  Responses  were  classified 
as  medium  frequency  if  the  respondent  indicated  very  fre- 
quently or  frequently  for  two  of  the  six  teaching  tasks  and 
were  classified  as  low  frequency  if  the  respondent  indicated 
very  frequently  or  frequently  for  zero  or  one  of  the  six 
teaching  tasks.  The  response  to  the  last  question  in 
Section  II,  Part  A,  was  used  to  determine  leverage  for  the 
combination  of  all  of  the  teaching  tasks.  The  gammas  for 
each  stage  of  the  evaluation  model  were  tested  for 


74 

significance  and  compared  with  gammas  from  the  other  stages 
for  strength  of  association. 

Summary 

This  chapter  has  presented  the  methods  which  were  used 
to  collect  and  analyze  data  to  investigate  the  relationship 
between  community  college  faculty  perceptions  of  frequency 
of  formal  and  informal  evaluation  and  their  perceptions  of 
their  own  effort  and  effectiveness.  Methods  used  to  collect 
and  analyze  data  relating  to  task  conceptions  as  varying 
along  an  active-inert  continuum  and  relating  to  the  relative 
effects  of  evaluation  in  a single  stage  of  the  evaluation 
model  were  also  described. 

Since  no  assumption  of  distribution  normality  could  be 
made,  a nonparametric  means  of  analysis  was  used.  Gamma,  a 
non-parametric  measure  of  association  between  two  ordinal- 
level  variables,  was  used  to  express  the  strength  of  the 
relationship  between  various  combinations  of  perceived  fre- 
quency of  evaluation,  perceived  effort  and  effectiveness, 
and  task  conceptions.  Each  gamma  was  tested  for  statistical 
significance  at  the  .05  level. 

Chapter  IV  presents  a description  of  the  responses  and 
the  results  of  the  analysis  of  these  response. 


CHAPTER  IV 


RESULTS 

This  project  was  designed  to  determine  whether  there 
was  a relationship  between  the  perceived  frequency  of 
evaluation  and  perceived  effort  in  relationship  to 
effectiveness  (leverage)  of  community  college  faculty. 
Ancillary  purposes  were  to  determine  whether  this 
relationship  was  stronger  for  tasks  that  were  classified  as 
more  active  along  the  active-inert  continuum,  whether  there 
was  a stronger  relationship  between  perceived  frequency  and 
leverage  in  a particular  stage  of  the  evaluation  process, 
and  whether  there  was  a stronger  relationship  between 
perceived  frequency  and  perceived  effort  or  between  perceived 
frequency  and  perceived  effectiveness. 

The  results  of  this  project  are  presented  in  this 
chapter.  The  results  are  presented  in  these  seven  sections: 
a description  of  the  project  sample  and  respondents, 
responses  relating  to  perceived  frequency  of  evaluation, 
responses  relating  to  leverage,  the  relationship  between 
frequency  of  evaluation  and  leverage,  results  of  classifying 
tasks  along  the  active-inert  continuum,  results  from 
analyzing  the  effort  and  effectiveness  components  of 
leverage  separately,  and  results  from  analyzing  perceived 
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frequency  of  evaluation  and  perceived  leverage  in  a single 
stage  of  the  evaluation  model  for  a combination  of  the  six 
teaching  tasks. 


Description  of  Project  Sample  and  Response 
The  results  are  based  on  5 clusters  that  were  selected 
from  the  28  community  colleges  in  Florida  using  a probabil- 
ity proportionate  to  size  sampling  technique,  with  100 
faculty  members  selected  from  each  of  the  5 colleges. 

The  five  clusters  chosen  for  the  sample  represented 
colleges  operating  in  urban  and  non-urban,  union  and  non- 
union, and  small  and  large  environments.  The  results  pre- 
sented in  Table  1 indicate  that  821  of  the  total  sample  of 
500  responded  after  multiple  follow-up  attempts,  with  a 
range  of  responses  from  individual  colleges  between  721  and 
91t. 


TABLE  1 


COLLEGES  INCLUDED  IN  THE  SAMPLE  AND  NUMBER 
AND  PERCENTAGE  OF  RESPONSES 


Number  Number  Percentage 

College  Sent  Returned  Returned 

Brevard  Too  51  5H 

Miami  Dade  100  79  79 

Tallahassee  100  88  88 


Florida 
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Palm  Beach  100  81  81 

Total  500  411  82t 

Note . The  500  faculty  surveyed  represent  12.6*  of  the  total 
population  of  community  college  faculty  in  Florida. 
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Table  2--Continued 


Frequently  or  Seldom  or 

Step  Very  Frequently  Occasionally  Never 

Task:  Professional 

Development 

11 

Criteria  Setting 

20 

SO 

13 

Communicating 

16 

48 

36 

Planning 

09 

45 

46 

Average 

14 

48 

38 

Task: Student 
Advising 

12 

40 

25 

Sampling 

14 

42 

44 

Communicating 

09 

38 

Planning 

09 

41 

SO 

Average 

14 

40 

46 

Task:  College 

Service 

Allocating 

09 

54 

37 

Criteria  Setting 

37 

Sampl i ng 

17 

53 

30 

Communicating 

15 

48 

37 

Planning 

14 

41 

45 

Average 

14 

49 

37 

Task: Maintenance 

Allocating 

27 

47 

26 

Criteria  Setting 

29 

27 

Sampling 

27 

46 

27 

Communicating 

43 

33 

Planning 

26 

44 

30 

Average 

26 

44 

30 

79 

An  analysis  of  the  frequency  distribution  reveals  that 
when  responses  for  all  tasks  and  all  steps  were  combined, 
only  191  of  the  respondents  indicated  that  they  were 
evaluated  either  formally  or  informally  on  a frequent  basis, 
and  an  average  of  361  of  the  respondents  selected  the  Never 
category  for  each  task.  Maintenance  factors,  a task  defined 
as  record  keeping,  office  hours,  and  other  routine 
activities,  had  the  highest  evaluation  activity  with  26%  of 
the  respondents  reporting  frequent  evaluation  activity  for 
this  task.  Student  advising  had  the  lowest  activity  with 
only  141  of  the  respondents  indicating  frequent  activity  and 
461  indicating  no  evaluation. 

The  step  of  the  evaluation  model  which  had  the  highest 
activity  across  all  tasks  was  criteria  setting,  with  an 
average  of  241  of  the  respondents  indicating  frequent 
evaluation  activity  in  this  step.  The  steps  indicating  the 
lowest  activity  were  allocating  (goal  setting)  and  planning 
for  improvement  with  respondents  indicating  frequent 
activity  only  171  of  the  time  in  each. 

Responses  Related  to  Leverage 

Two  measures  of  leverage  were  calculated  from  different 
items  on  the  questionnaire.  In  questions  relating  to  the 
first  measure,  the  Faculty  Assessment  of  Leverage  (FAL), 
faculty  were  asked  to  describe  their  level  of  effort  in 
relationship  to  the  payoff  in  terms  of  results  for  each  of 
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Che  six  teaching  tasks.  If  the  respondent  indicated  that 
the  payoff  was  greater  than  effort,  the  response  was  placed 
in  the  category  indicating  leverage  greater  than  or  equal  to 
one.  If  the  respondent  indicated  that  payoff  was  less  than 
effort,  the  response  was  placed  in  the  category  indicating 
leverage  less  than  one. 

In  two  questions  relating  to  the  Composite  Assessment 
of  Leverage  (CAL),  faculty  described  their  present  effort 
and  effectiveness  relative  to  their  effort  and  effectiveness 
earlier  in  their  careers.  A ratio  of  the  responses  to  the 
two  questions  was  used  to  place  the  respondents  in  the 
leverage  greater  than  or  equal  to  one  category  or  the 
leverage  less  than  one  category. 

The  results  are  presented  in  Table  3.  The  CAL  measure 
was  less  variable  than  the  FAL  measure.  At  least  701  of  all 
respondents  reported  leverage  greater  than  or  equal  to  one 
on  the  CAL  measure.  The  lowest  proportion  of  faculty  were 
in  the  leverage  greater  than  or  equal  to  one  category  for 
the  tasks  of  classroom  teaching  and  maintenance  factors 
using  both  the  CAL  and  FAL  measures.  The  highest  proportion 
of  faculty  were  in  the  leverage  greater  than  or  equal  to  one 
category  for  the  task  of  student  advising  using  both  the  CAL 
and  FAL  measures. 

For  the  average  of  all  tasks,  60%  of  the  respondents 
indicated  leverage  greater  than  or  equal  to  one  using  the 
FAL  measure  of  leverage  and  794  using  the  CAL  measure.  The 
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proportion  for  the  CAL  measure  was  consistently  251  to  3S1 
larger  than  for  the  FAL  measure.  The  range  for  the  FAL 
measure  was  52%  to  73t  and  701  to  881  for  the  CAL  measure. 


TABLE  3 

PROPORTION  OF  RESPONDENTS  INDICATING  LEVERAGE 
GREATER  THAN  OR  EQUAL  TO  1 


Task 

Classroom  Teaching 
Curriculum  Development 
Professional  Development 
Student  Advising 
College  Service 
Maintenance  Factors 


Measure  of  Leverage 

FAL CAL 

.52  .70 

.66  .77 

.60  .79 

.73  .88 

. S9  .80 

.54  .73 


The  Relationship  Between  Perceived  Frequency 

To  answer  question  number  1 of  the  problem  statement 
and  to  determine  whether  there  was  a statistically 
significant  association  (g  < .05)  between  perceived 
frequency  of  evaluation  and  perceived  leverage  for  each  of 
the  six  teaching  tasks,  respondents  were  categorized 
according  to  perceived  frequency  of  evaluation  and  leverage 
using  both  the  FAL  and  CAL  measure  of  leverage.  Respondents 
selecting  either  Frequently  or  Very  Frequently  for  two  or 


stages  of  the  evaluation  model 


classified  as  high 


those  selecting 


for  only 


frequency  for  that  task, 
stage  of  the  evaluation  model  were  classified  as  medium 
frequency,  and  those  selecting  them  in  no  stages  were 
categorized  as  low  frequency.  The  number  of  respondents  in 
each  category  is  reported  in  Table  4 for  the  FAL  measure  of 
leverage  and  in  Table  5 for  the  CAL  measure  of  leverage. 

TABLE  4 

FREQUENCY  OF  EVALUATION  AND  LEVERAGE  BY  NUMBER  OF 
RESPONDENTS  USING  THE  FAL  MEASURE  OF  LEVERAGE 

Frequency  of  Evaluation 


Leverage  Low  Medium  High 

Task:  Classroom 

Teaching 

Less  than  1 96  44  59 

Equal  to  l 128  52  52 

Gamma  = -.25  z = 2.46 

Task:  Curriculum 

Development 

Less  than  1 100  16  20 

Equal  totian  °r  164  24  87 

Gamma  - +.33  z - 2.61 

Task:  Professional 

Development 

Less  than  1 118  11  38 

Greater  than  or 

Equal  to  1 148  52  44 

Gamma  = +.12  z = .841 


Table  j- -Continued 


Leverage  Low Medium High 

Task:  Student 

Advising 

Less  than  1 92  20  12 

Greater  than  or 

Equal  to  1 218  22  60 

Gamma  e +.09  z - .591 

Task:  College 

Less  than  1 151  8 8 

Equal  to  l 132  28  24 

Gamma  = +.29  z = 2.051 

Task:  Maintenance 

Less  than  1 150  8 31 

Equal  to  1 128  22  72 

Gamma  = +.06  z - .432 


TABLE  5 


FREQUENCY  OF  EVALUATION  AND  LEVERAGE  BY  NUMBER  OF 
RESPONDENTS  USING  THE  CAL  MEASURE  OF  LEVERAGE 


Leverage  Low  Medium  High 

T^k:  Classroom  “ 

Teaching 

Less  than  1 58  28  40 

Greater  than  or 

Equal  to  1 187  74  24 

Gamma  = -.38  z = 2.688 

Task:  Curriculum 

Development 

Equal  to  1 188  38  90 

Gamma  = t.36  z = 2.448 

Task:  Professional 

Development 

Less  than  1 42  14  31 

Greater  than  or 

Equal  to  1 227  18  79 

Gamma:  <-.06  z - .491 

T^H  Student 

Less  than  1 38  19  18 

Greater  than  or 


240 


Table  5--Continued 


Leverage 

Task:  College  Servii 


Gamma  » +.26 
Task:  Maintenance 


Frequency  of  Evaluation 

Low Medium High 


56 


21 


232 


z = .146 


The  values  of  gamma,  a measure  of  strength  of 
association  between  perceived  frequency  of  evaluation  and 
leverage  are  summarized  in  Table  6,  for  both  the  CAL  and  the 
FAL  measures  of  leverage.  Although  the  gammas  using  the  FAL 
and  CAL  measures  of  leverage  are  different,  the  gammas  using 
both  measures  are  statistically  significant  at  the  .OS  level 
for  the  same  tasks  (classroom  teaching,  curriculum 
development,  and  college  serv 


'ice) . 


TABLE  6 


*2  < .05,  two-tailed. 


♦ .33* 

♦ .12 


signifii 


!!mi 


iignifit 


TABLE  7 

NUMBER  OF  RESPONDENTS  CLASSIFYING  EACH  TASK  AS 
ACTIVE,  MODERATELY  ACTIVE,  AND  INERT 


Classroom  Teaching 
Curriculum  Development 
Professional  Development 
Student  Advising 
College  Service 


Inert 

247 

144 


100 

227 


92 


139 

113 

100 

72 


TABLE  8 

CLASSIFICATION  OF  TASKS  ALONG  THE 
ACTIVE-INERT  CONTINUUM 

Tasl(  Median  Response  Category 

Teaching 

Curriculum  Development 
Professional  Development 
Student  Advising 
College  Service 


GAMMAS  FOR  THE  RELATIONSHIP  BETWEEN  PERCEIVED  FREQUENCY 
OF  EVALUATION  AND  LEVERAGE  FOR  ACTIVE,  MODERATELY 
ACTIVE,  AND  INERT  TASKS 

**a 

*"4 


Results  from  Analyzing  the  Effort 


analysis  of 


«sjs;s 


'sis;*1 


f c;“ss  **”- 

.,  Evaluation  and 


in  the  two  highest  response  categories  [More  Effective/Much 
More  Effective  and  Somewhat  Harder/Much  Harder)  as  high 
effectiveness  or  effort,  by  classifying  responses  in  the 
middle  response  category  (About  as  Effective  or  About  as 
Hard)  as  medium  effectiveness  or  effort,  and  by  classifying 
responses  in  the  lowest  two  response  categories  (Much  Less 
Effective/Less  Effective  and  Not  Nearly  As  Hard/Not  Quite  as 
Hard)  as  low  effort  or  effectiveness.  Responses  relating  to 
frequency  of  evaluation  were  classified  into  high,  medium, 
and  low  categories  using  the  same  techniques  previously  used 
in  the  study. 

For  the  task  of  classroom  teaching,  although  there  was 
a slightly  positive  relationship  between  perceived  frequency 
of  evaluation  and  perceived  effectiveness,  there  was  a 
higher  positive  and  significant  relationship  between  per- 
ceived frequency  of  evaluation  and  perceived  effort.  Since 
leverage  was  defined  as  the  ratio  of  perceived  effectiveness 
to  perceived  effort,  this  significant  positive  association 
with  effort  is  the  reason  for  the  significant  negative 
association  between  perceived  frequency  of  evaluation  and 
perceived  leverage  for  this  task. 

For  the  tasks  of  curriculum  development  and  college 
service,  a significant  and  moderately  positive  relationship 
was  found  between  perceived  frequency  of  evaluation  and 
perceived  effectiveness.  There  was  no  significant 
relationship  between  perceived 


frequency  of  evaluation  and 
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perceived  effort  for  either  task.  For  these  tasks,  the 
positive  association  between  perceived  frequency  of  evalua- 
tion and  perceived  effectiveness  is  the  reason  for  the 
significant  positive  association  between  perceived  frequency 
of  evaluation  and  perceived  leverage. 

Results  from  Analyzing  Perceived  Frequency 
of  Evaluation  and  Perceived  Leverage  in  a 
Single  Stage  of  the  Evaluation  Model 

Most  of  the  analysis  in  this  study  involved  investi- 
gating the  relationships  between  perceived  frequency  of 
evaluation  and  leverage  for  each  of  the  six  teaching  tasks 
using  responses  to  all  five  steps  in  the  evaluation  model. 
However,  an  ancillary  purpose  of  the  project  was  to  deter- 
mine whether  there  was  a stronger  relationship  between  per- 
ceived frequency  of  evaluation  in  any  single  stage  of  the 
evaluation  model  and  leverage  for  a combination  of  the  six 
teaching  tasks. 

For  each  step  of  the  evaluation  model,  an  individual 
respondent  was  classified  as  high  perceived  frequency  of 
evaluation  for  that  step  if  the  response  was  either  Fre- 
quently or  Very  Frequently  for  three  or  more  tasks,  was 
classified  as  medium  frequency  if  the  response  was  either 
Frequently  or  Very  Frequently  for  two  tasks,  and  was  classi- 
fied as  low  frequency  if  the  response  was  either  Frequently 
or  Very  Frequently  for  zero  or  one  of  the  six  tasks.  A com- 
posite measure  of  leverage  was  obtained  from  a question  on 


pip  Ml 


94 

perceived  leverage  in  a single  stage  of  the  evaluation  model 
for  a combination  of  the  six  tasks.  These  results  are 
presented  in  Table  12. 

TABLE  12 

GAMMAS  FOR  PERCEIVED  FREQUENCY  OF  EVALUATION  AND 
LEVERAGE  IN  A SINGLE  STEP  OF  THE  EVALUATION 
MODEL  FOR  A COMBINATION  OF  ALL  TASKS 


Step 

Gamma 

Allocating 

♦ .28* 

Criteria  Setting 

+ .13 

Sampling 

+ .39* 

Communicating 

+ .19 

Planning  for  Improvement 

+ .42* 

*£  < .05,  two-tailed. 


Positive  and  significant  associations  between  perceived 
frequency  of  evaluation  and  leverage  for  a combination  of 
all  tasks  were  found  for  the  steps  of  allocating,  sampling, 
and  planning  for  improvement.  There  was  no  significant 
relationship  between  perceived  frequency  of  evaluation  and 
leverage  for  the  steps  of  criteria  setting  and 
communicating. 

A summary  of  these  results  and  conclusions  and  recom- 
mendations drawn  from  these  results  are  presented  in  Chapter 
V. 


CHAPTER 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 
Summary 

Project  Rationale  and  Design 

Faced  with  declining  enrollments,  declining  financial 
resources,  an  increasingly  "tenured-in,"  less-mobile,  and 
aging  faculty,  and  an  increasing  public  outcry  for  account- 
ability, community  colleges  have  shown  an  increased  interest 
in  faculty  evaluation  as  a possible  solution  to  these 
problems.  Most  of  this  attention  to  evaluation  has  been 
focused  on  methods  and  procedures  for  evaluating  faculty. 
Few  researchers  have  concentrated  on  the  effects  of 
evaluation  on  faculty. 

One  of  the  first  comprehensive  studies  was  reported  by 
Dornbusch  and  Scott  in  1975.  They  concluded  that,  based  on 
studies  conducted  in  schools,  there  was  a relationship 
between  frequency  of  evaluation  and  increases  in  teacher 
satisfaction.  Dornbusch  and  Scott  (1975)  also  suggested 
that  tasks  vary  along  an  active-inert  continuum  based  on 
clarity,  predictability,  and  efficacy  and  that  performers' 
preferences  for  work  arrangements  and  supervisory  behavior 
vary  according  to  their  conceptions  of  various  tasks. 
Dornbusch  and  Scott  also  developed  a four-step  model 


of  the  evaluation  process  which  included  the  steps  of  allo- 
cating (task  assignment  and  goal  setting),  criteria  setting 
(establishing  measurements),  sampling  (collecting  informa- 
tion), and  appraisal  (performing  the  evaluation). 

Natriello  (1984)  expanded  the  model  to  include  the 
additional  steps  of  communication  and  planning  for  improve- 
ment. Building  on  the  work  of  Dornbusch  and  Scott  (197S), 
he  studied  the  effects  of  evaluation  on  the  effort  and 
effectiveness  of  elementary  school  teachers.  He  developed 
the  concept  of  leverage  to  express  the  ratio  between  per- 
ceived effectiveness  and  perceived  effort.  He  found  a 
strong  positive  relationship  between  perceived  frequency  of 
evaluation  and  leverage  and  also  found  that  perceived  fre- 
quency of  evaluation  was  related  to  increases  in  perceived 
effectiveness  rather  than  to  decreases  in  perceived  effort. 

To  test  the  applicability  of  the  Dornbusch  and  Scott 
(1975)  and  Natriello  (1984)  research  to  community  colleges, 
this  study  was  designed  to  determine  the  relationship 
between  perceived  frequency  of  formal  and  informal  evalua- 
tion and  perceived  effectiveness  in  relationship  to  effort 
(leverage)  for  community  college  faculty.  The  main  purpose 
of  the  study  was  to  determine  whether  there  was  a 
statistically  significant  association  at  the  .05  level 
between  perceived  frequency  of  evaluation  and  leverage  for 
each  of  these  six  teaching  tasks:  classroom  teaching, 
curriculum  development,  professional  development,  student 
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advising,  college  service,  and  maintenance  factors. 
Ancillary  purposes  of  the  project  were  to  determine  whether 
this  association  was  stronger  for  tasks  faculty  perceived  as 
active  or  for  tasks  they  perceived  as  inert,  whether  there 
was  a stronger  relationship  between  perceived  frequency  of 
evaluation  and  perceived  effectiveness  or  between  perceived 
frequency  of  evaluation  and  perceived  effort,  and  whether 
there  was  a stronger  relationship  between  perceived  fre- 
quency of  evaluation  and  leverage  in  any  one  stage  of  the 
six-step  evaluation  model  for  a combination  of  all  six 
teaching  tasks. 

A survey  research  design  was  adopted  for  this  project, 
and  a questionnaire  was  developed  to  determine  perceived 
frequency  of  evaluation  and  perceived  effectiveness  and 
effort  for  each  of  the  six  teaching  tasks  and  to  determine 
the  placement  of  each  task  along  the  active-inert  continuum. 
The  population  was  all  full-time  faculty  in  the  28  community 
colleges  in  Florida.  A sample  of  five  clusters  of  colleges 
was  drawn  using  a probability  proportionate  to  size  sampling 
technique,  and  100  faculty  members  were  randomly  selected 
from  each  of  the  five  clusters.  After  multiple  follow-up 
attempts,  821  of  the  sample  of  500  responded  to  the  mail-out 
questionnaire. 

The  responses  to  the  Likert-scale  categories  for  each 
question  were  analyzed  by  constructing  a summary  measure  of 
frequency  of  evaluation  and  two  summary  measures  of 
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For  the  tasks  of  curriculum  development  and  college 
service,  there  was  a statistically  significant  association 
between  perceived  frequency  of  evaluation  and  perceived 
effectiveness  and  no  statistically  significant  association 
between  perceived  frequency  of  evaluation  and  perceived 
effort.  For  the  task  of  classroom  teaching,  there  was  a 
statistically  significant  association  between  perceived 
frequency  of  evaluation  and  perceived  effort  and  no 
statistically  significant  association  between  perceived 
frequency  of  evaluation  and  perceived  effectiveness. 

Question  4.  In  which  stages  of  the  evaluation  process 
did  perceived  frequency  of  evaluation  have  the 
strongest  association  with  perceived  effectiveness  in 
relationship  to  effort  (leverage)  for  a combination  of 
the  six  teaching  tasks? 

The  strongest  significant  relationships  between 
perceived  frequency  of  evaluation  and  leverage  for  a 
combination  of  the  six  teaching  tasks  were  found  in  the 
stages  of  allocating,  sampling,  and  planning  for 
improvement.  There  was  no  significant  relationship  between 
perceived  frequency  of  evaluation  and  leverage  for  the 
stages  of  criteria  setting  and  communicating. 

Perceived  frequency  of  evaluation  and  leverage.  No 
significant  association  was  found  at  the  .05  level  between 


perceived  frequency  of  evaluation  and  leverage  using  either 
the  CAL  and  FAL  measures  of  leverage  for  the  tasks  of 
professional  development,  student  advising,  or  maintenance 
factors.  A significant,  positive  association  was  found  for 
the  tasks  of  curriculum  development  and  college  service,  and 
a significant,  negative  association  was  found  for  the  task 
of  classroom  teaching. 

For  the  task  of  college  service  and  curriculum 
development,  there  was  a moderately  positive  relationship 
between  perceived  frequency  of  evaluation  and  leverage 
(gamma  = .33  using  FAL  and  .36  using  CAL).  A similar 
relationship  was  found  for  the  task  of  college  service 
(gamma  = .29  using  FAL  and  .26  using  CAL).  This  would 
confirm  the  hypotheses  of  Dornbusch  and  Scott  (1975)  and 
Natriello  (1984). 

For  the  task  of  classroom  teaching,  there  was  a 
moderately  negative  association  between  perceived  frequency 
of  evaluation  and  leverage  (gamma  = -.25  using  FAL  and  -.38 
using  CAL).  These  results  are  contrary  to  the  hypotheses  of 
Dornbusch  and  Scott  (1975)  and  Natriello  (1984). 

An  analysis  of  these  results  indicated  that  there  might 
be  positive  benefits  in  terms  of  perceived  effectiveness  in 
relationship  to  effort  from  increasing  evaluation  activity 
for  the  tasks  of  curriculum  development  and  college  service. 
In  contrast,  there  might  be  a negative  effect  from 
increasing  evaluation  activity  for  the  task  of  classroom 
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tasks  reported  in  his  research,  it  is  possible  that  his 
intended  definition  and  the  one  used  in  this  study  were  dif- 

Support  for  the  idea  that  task  conceptions  affect  the 
relationship  between  perceived  frequency  of  evaluation  and 
leverage  was  even  stronger  when  the  data  for  active  tasks, 
moderately  active  tasks,  and  inert  tasks  were  combined  for 
the  tasks  in  each  category,  and  gammas  were  calculated  for 
the  relationship  between  perceived  frequency  of  evaluation 
and  leverage  using  the  combined  data.  The  gamma  was  .46  for 
active  tasks,  .18  for  moderately  active  tasks,  and  -.39  for 
inert  tasks.  Although  Natriello  (1984)  hypothesized  that 
very  inert  tasks  would  show  no  relationship  between  per- 
ceived frequency  of  evaluation  and  leverage,  in  this  study 
there  was  a significant,  negative  relationship  between 
perceived  frequency  of  evaluation  and  leverage  for  inert 

The  analysis  of  the  data  provided  an  affirmative  answer 
to  the  problem  statement  which  asked  whether  there  was  a 
stronger  relationship  between  perceived  frequency  of  evalua- 
tion and  leverage  for  tasks  that  were  more  active  than  for 
tasks  that  were  less  active  or  inert. 


Relationship  between  perceived  frequency  of  evaluation 
effort  and  effectiveness  components  of 
s designed  to 
effectiveness 


CAL  measure  of  leverage 


analysis 


the  effort 
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components  of  leverage  separately.  The  purpose  of  this 
analysis  was  to  answer  the  question  in  the  problem  statement 
which  asked  whether  there  was  a stronger  association  between 
perceived  frequency  of  evaluation  and  perceived 
effectiveness  or  between  perceived  frequency  of  evaluation 
and  perceived  effort  for  tasks  where  there  was  a significant 
association  at  the  .05  level  between  perceived  frequency  of 
evaluation  and  leverage. 

There  was  no  significant  association  between  perceived 
frequency  of  evaluation  and  perceived  effectiveness  for  the 
task  of  classroom  teaching,  but  there  was  a significant  and 
moderately  positive  association  between  perceived  frequency 
of  evaluation  and  perceived  effort  (gamma  = .32).  This  sug- 
gests that  increases  in  the  frequency  of  evaluation  for  this 
task  were  related  to  increases  in  effort  but  not  to 
increases  in  effectiveness,  thus  explaining  the  negative 
association  between  perceived  frequency  of  evaluation  and 
leverage. 

For  the  tasks  of  curriculum  development  and  college 
service,  there  was  a positive  association  between  perceived 
frequency  of  evaluation  and  perceived  effectiveness  for  both 
tasks  (gammas  - .21  and  .22,  respectively)  and  no 
significant  association  at  the  .OS  level  between  perceived 
frequency  of  evaluation  and  perceived  effort.  This 
indicated  that  increases  in  the  frequency  of  evaluation  were 
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related  to  increases  in  perceived  effectiveness  for  these 

The  conclusion  was  that  for  tasks  where  there  was  a 
significant  negative  association  between  perceived  frequency 
of  evaluation  and  perceived  leverage,  this  was  the  result  of 
a stronger  association  between  perceived  frequency  of  evalu- 
ation and  perceived  effort  than  between  perceived  frequency 
of  evaluation  and  perceived  effectiveness.  For  tasks  where 
there  was  a significant  positive  association  between  per- 
ceived frequency  of  evaluation  and  perceived  leverage,  this 
was  the  result  of  a stronger  association  between  perceived 
frequency  of  evaluation  and  perceived  effectiveness  than 
between  perceived  frequency  of  evaluation  and  perceived 
effort.  This  confirmed  Natriello's  ( 1984)  research  which 
showed  that  increases  in  perceived  frequency  of  evaluation 
were  associated  with  increases  in  effectiveness,  except  that 
his  data  included  no  negative  associations  for  comparison 
with  the  data  from  this  study. 

Perceived  frequency  of  evaluation  and  leverage  in  a 
single  stage  of  the  evaluation  model.  When  responses  were 
analyzed  for  a single  step  of  the  evaluation  model  for  a 
combination  of  all  tasks,  there  were  significant 
associations  at  the  .05  level  between  perceived  frequency  of 
evaluation  and  leverage  for  the  steps  of  allocating, 
sampling,  and  planning  for  improvement,  with  gammas  of  .28, 
.39,  and  .42,  respectively.  There  were  no  significant 
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associations  at  the  .05  level  between  perceived  frequency  of 
evaluation  and  perceived  leverage  for  the  tasks  of  criteria 

The  question  in  the  problem  statement  which  asked 
whether  there  was  a stronger  association  between  perceived 
frequency  of  evaluation  and  leverage  in  any  single  stage  of 
the  evaluation  model  was  answered  affirmatively.  There  was 
a stronger  relationship  between  increases  in  perceived 
frequency  of  evaluation  and  leverage  for  the  stages  of 
allocating,  sampling,  and  planning  for  improvement  than  for 
the  stages  of  criteria  setting  and  communicating. 

Conclusions  and  Implications 

An  analysis  of  the  data  collected  in  this  study 
suggests  that  the  leverage  of  community  college  faculty 
might  be  increased  for  certain  tasks  through  more  frequent 
evaluations  of  those  tasks.  This  might  be  of  great  impor- 
tance to  community  college  administrators  striving  to  find 
ways  to  increase  the  effectiveness  of  their  faculty.  Based 
on  these  data,  those  who  perceive  evaluation  as  a control 
mechanism  and  a threat  to  the  intrinsic  motivation  and 
academic  and  professional  freedom  of  faculty  might  consider 
the  positive  effects  of  increased  evaluation  in  increasing 
faculty  control  over  their  own  teaching  tasks. 

Although  more  research  is  needed  to  determine  why  there 
was  a positive  relationship  between  perceived  frequency  of 


task  of  classroom  teaching  to  be  highly  inert  and  that  there 
was  a significant  negative  association  between  perceived 
frequency  of  evaluation  and  leverage.  When  the  wide  variety 
of  student  abilities  and  backgrounds,  the  plethora  of  new 
research  on  topics  such  as  learning  styles  and  testing 
theory,  and  the  increasing  choices  provided  by  developments 
in  educational  technology  are  considered,  it  is  almost 
inconceivable  that  faculty  consider  classroom  teaching  to  be 
high  on  predictability,  clarity,  and  efficacy.  Serious 
negative  consequences,  both  for  the  work  experiences  of 
faculty  and  the  learning  experiences  of  students,  might 
result  from  this  finding. 

Based  on  data  collected  in  this  study,  the  increased 
emphasis  which  is  being  placed  on  evaluation  in  many  col- 
leges might  have  negative  effects  on  the  perceptions  of 
faculty  of  their  own  effort  and  effectiveness  for  the  task 
of  classroom  teaching.  This  would  create  a situation  where 
classroom  teaching,  the  most  important  task  of  community 
college  faculty,  would  be  considered  by  faculty  to  be  not 
only  predictable  and  unchallenging  but  also  to  be  a task 
where  payoff  in  terms  of  results  is  less  than  effort.  Com- 
munity college  administrators  striving  to  improve  classroom 
teaching  might  want  to  consider  placing  more  emphasis  on 
staff  development,  and  less  on  staff  evaluation,  to  assist 
faculty  in  gaining  new  skills  or  new  perspectives  to  move 
classroom  teaching  back  into  the  active  task  category. 
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Increased  evaluation  activity  appears  not  to  be  the  right 
solution  to  improving  classroom  teaching. 

An  analysis  of  the  data  also  indicated  that  increasing 
certain  types  of  evaluation  may  be  more  effective  than  other 
types  in  increasing  leverage.  There  was  no  association 
between  perceived  frequency  of  evaluation  and  leverage  for 
the  steps  of  criteria  setting  and  communicating,  and  there 
was  a positive  association  for  all  other  steps.  It  is 
possible  that  faculty  responding  to  the  questionnaire 
equated  criteria  setting  and  communicating  to  the  formal 
college  evaluation  form  and  the  required  yearly  supervisory 
evaluation.  If  this  were  true,  these  results  might  reflect 
a dissatisfaction  with  a process  that  is  often  considered  to 
be  too  infrequent  and  arbitrary.  The  steps  of  allocating, 
sampling,  and  planning  for  improvement  in  the  evaluation 
model  might  be  used  more  frequently  to  increase  faculty 
perceptions  of  effectiveness  in  relationship  to  effort. 

Recommendations 

Although  much  of  the  research  of  Dornbusch  and  Scott 
(1975)  and  Natriello  (1984)  was  supported  by  this  study  and 
additional  data  are  now  available  with  which  to  study  the 
impact  of  evaluation  processes  on  faculty,  it  is  not  a set 
of  final  answers.  Additional  data  collected  by  other 
methods  and  in  other  settings  are  needed  before  conclusions 
can  be  reached  about  the  role  evaluation  processes  might 
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play  in  the  operation  of  effective  community  colleges.  The 
following  are  recommendations  for  further  research. 

1.  The  data  collected  in  this  study  and  the  studies 
of  Dornbusch  and  Scott  (197S)  and  Natriello  (1984)  were 
based  on  teacher  and  faculty  perceptions.  Experimental 
confirmation  of  these  findings  is  necessary  if  the  direction 
of  causation  in  the  relationship  between  perceived  frequency 
of  evaluation  and  perceived  effectiveness  in  relationship  to 
effort  is  to  be  certain. 

2.  Other  variables  should  be  examined  when  determin- 
ing which  tasks  show  the  strongest  relationship  between  per- 
ceived frequency  of  evaluation  and  leverage.  In  particular, 
an  analysis  of  the  data  collected  in  this  study  intuitively 
indicates  that  the  visibility  of  task  behavior  may  be  a 
factor  to  be  considered. 

3.  Data  should  be  collected  and  analyzed  from  the 
same  population  using  a personal  interview  technique  to 
determine  perceived  frequency  of  evaluation  and  perceived 
effort  and  effectiveness.  Data  on  frequency  of  evaluation, 
effort,  and  effectiveness  should  also  be  obtained  from 
supervisors  and  compared  with  faculty  data. 

4.  Organizational  climate  should  be  considered  as  a 
possible  intervening  variable  in  the  relationship  between 
perceived  frequency  of  evaluation  and  leverage. 


5.  Research  is  needed  to  determine  why  classroom 
teaching  is  considered  inert  and  to  determine  the  implica- 
tions of  this  task  conception  on  teachers  and  on  students. 

6.  The  effects  of  supervisory  training  in  performance 
appraisal  and  performance  coaching  techniques  on  the  rela- 
tionship between  perceived  frequency  of  evaluation  and 
leverage  should  be  studied. 

In  conclusion,  an  analysis  of  the  data  from  this 
research  suggests  that  evaluation  may  affect  the  perceptions 
of  community  college  faculty  of  their  own  effort  and 
effectiveness.  For  some  tasks  the  effect  is  positive  and 
for  others  it  is  negative,  and  some  types  of  evaluation 
activity  have  more  effect  than  others.  At  the  very  least, 
it  is  a step  towards  silencing  the  opponents  of  faculty 
evaluation  who  have  slowed  progress  in  the  study  of 
evaluation,  and  it  provides  community  college  administrators 
with  information  that  might  be  useful  in  designing  both 
evaluation  systems  and  faculty  and  supervisory  development 
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